-- Logs begin at Thu 2022-06-30 12:26:20 UTC, end at Thu 2022-06-30 13:29:33 UTC. -- Jun 30 12:51:49.082092 dv-cinder-812685-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 12:51:49.095288 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 12:51:49 2022] *** Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: nodename: dv-cinder-812685-14-iscsi Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: machine: x86_64 Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: clock source: unix Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: pcre jit disabled Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: detected number of CPU cores: 4 Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: current working directory: / Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: detected binary path: /usr/bin/uwsgi-core Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: your processes number limit is 47680 Jun 30 12:51:49.097326 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: your memory page size is 4096 bytes Jun 30 12:51:49.097901 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: detected max file descriptor number: 2048 Jun 30 12:51:49.097901 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: lock engine: pthread robust mutexes Jun 30 12:51:49.097901 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: thunder lock: enabled Jun 30 12:51:49.097901 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 12:51:49.097901 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 12:51:49.141056 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: Python main interpreter initialized at 0x561c245909c0 Jun 30 12:51:49.141056 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: python threads support enabled Jun 30 12:51:49.141056 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: your server socket listen backlog is limited to 100 connections Jun 30 12:51:49.141056 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: your mercy for graceful operations on workers is 90 seconds Jun 30 12:51:49.141373 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 12:51:49.141484 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: *** Operational MODE: preforking *** Jun 30 12:51:49.141864 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: *** uWSGI is running in multiple interpreter mode *** Jun 30 12:51:49.141864 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: spawned uWSGI master process (pid: 80089) Jun 30 12:51:49.143819 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: spawned uWSGI worker 1 (pid: 80101, cores: 1) Jun 30 12:51:49.143819 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: spawned uWSGI worker 2 (pid: 80102, cores: 1) Jun 30 12:51:49.143819 dv-cinder-812685-14-iscsi devstack@keystone.service[80089]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 12:51:49.142082 dv-cinder-812685-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 12:51:54.486997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:51:54.486997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 30 12:51:54.487822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:51:54.487822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: from cryptography.utils import int_from_bytes Jun 30 12:51:54.488223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:51:54.488223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 30 12:51:54.488325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 12:51:54.488325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: from cryptography.utils import int_from_bytes Jun 30 12:51:54.535805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 12:51:54.535973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 12:51:54.536055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 12:51:54.536132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 12:51:54.536192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] command line args: [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 12:51:54.536257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] command line args: [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 12:51:54.536319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 12:51:54.536380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 12:51:54.536539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ================================================================================ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 12:51:54.536618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ================================================================================ {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 12:51:54.536725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.536809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] admin_token = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.536997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_dir = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] config_dir = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_source = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] config_source = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] debug = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] debug = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.537969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.538999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_config_append = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_config_append = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_dir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_dir = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_file = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.539889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.540838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.541930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.542881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] publish_errors = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] publish_errors = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.543945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] standard_threads = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.544906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] standard_threads = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_journal = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] use_journal = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_json = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] use_json = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_stderr = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.545990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] use_stderr = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.546067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_syslog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.546137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] use_syslog = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.546209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.546283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 12:51:54.546503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.546644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.546785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.546863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.546932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.547944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.548952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.549996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.550896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.551992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.552982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.553895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.554899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.555986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.556989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.557983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.558977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.559893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.560990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.561903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.562932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.563949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.564955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.565951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.566941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.567980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.568901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.569922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.external = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.570996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.external = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.password = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.token = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.571939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.password = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] auth.token = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.572954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.573970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.574898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.575990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.576993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.577897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.578973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.579932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.580909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.581916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.582951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.583969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.584979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.585994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.586910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.587925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.588951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.589955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.590889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.591894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.592998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.593995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.594931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.595821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.596924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.597923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.598902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.599907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.600912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.601922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.602896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] role.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] role.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.603918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.604936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.605941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.606974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.607983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.608991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.609902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.610956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.611937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.612937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.613826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.614955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.connection = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.615902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.616923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.617957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 12:51:54.618808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.618808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.619224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.619224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 12:51:54.619431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80101) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 12:51:54.648646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.648883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.649111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.653470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.653702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.653934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.657534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.662370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.685503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.685726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.687993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.688202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.832062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.832257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.880121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.880314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.880998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.881233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.943975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.944104 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.944192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.944256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.944406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.968992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.969944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.980641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.980768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.993537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.993675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.993907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.993992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80101) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:54.994933 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 12:51:55.012014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.012224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:51:55.012348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.012555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.012702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:51:55.012840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:51:55.012922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 12:51:55.013217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.013834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.014014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.024743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.024869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.025622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.033690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.033818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.036947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.037075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.037229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.037313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.037475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.037613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.037750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.037831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.038880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.039611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.065758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.065883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.067382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.067509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.067656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.067744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.072827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.072953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.073090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.073170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.073357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.073482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.080764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.080890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.081027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.081124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.085503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.085627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.085765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.085845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.085965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.086849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.087051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.087178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.087316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.087410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.106014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.106142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.106263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.106374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.110855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.110982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.114495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.114621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.114756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.114854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.119855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.119983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.123425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.123551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.127178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.127322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.127462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.127581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.127696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.127773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.128684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.147452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.147579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.147707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.147789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.147964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.148088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.148220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.148302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.156055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.156269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.157746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.157912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.158004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.158078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.161178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.161320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.161458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.161532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.161708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.161834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.161970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.162062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.162215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.162305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.175444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.175636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.175734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.175934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.176830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.177010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.178301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.178625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.195954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.196373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.199195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.199615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.202463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.202917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.204145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.204874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.205904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.206171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.206456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.206579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.206785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.207060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.213928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.214685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.215834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.216309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.218873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.219164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.219439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.219562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.219790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.219916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.220259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.220537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.229034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.229313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.229583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.229727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.229865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.230127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.237592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.237866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80101) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 12:51:55.238148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.238901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.239960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.240828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.241030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.241201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.241328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.241538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80101) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 12:51:55.278473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c245909c0 pid: 80102 (default app) Jun 30 12:51:55.279083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561c245909c0 pid: 80101 (default app) Jun 30 12:51:55.281931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c51d-3e84-4677-87d5-63d17b4f2045 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:51:55.282250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c51d-3e84-4677-87d5-63d17b4f2045 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:51:55.282539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0c51d-3e84-4677-87d5-63d17b4f2045 None None] PATH_INFO: `/v3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:51:55.286753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Thu Jun 30 12:51:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:03.016435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:03.017031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:03.017031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:03.021330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:03.028204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:03.028959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:03.029103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:03.099608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: INFO dbcounter [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] Registered counter for database keystone Jun 30 12:52:03.124221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG oslo_db.sqlalchemy.engines [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80102) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 12:52:03.135567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writer thread running {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 12:52:03.323807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80102) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 12:52:03.324149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80102) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 12:52:03.379241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:03.380865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:03.514950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 12:52:03.740864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:03.816928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: INFO dbcounter [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] Registered counter for database keystone Jun 30 12:52:03.838580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG oslo_db.sqlalchemy.engines [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80101) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 12:52:03.848406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writer thread running {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 12:52:03.967929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7fecb-5eb3-4283-b020-8ff6c2ee0ae7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:03.973358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:03.974037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:03.974300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:03.974571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:03.984489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:04.042992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING keystone.common.rbac_enforcer.enforcer [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 12:52:04.043591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ea3f8-d735-4b7d-b8a4-fd59f771617b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:04.053558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Thu Jun 30 12:52:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:52:04.060081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:04.075450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0c56a-4c98-4851-bad8-5d4928de1f7c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:04.080761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:04.081896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:04.082177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:04.082468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] PATH_INFO: `/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:04.084095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] RBAC: Authorizing `identity:get_project(project_id=288f97ff4fc34a4e979d3c926f4a9c7f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:04.137708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 12:52:04.138041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6eb95-fe04-44e8-9187-b1c86fd2c0d0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:04.142755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Thu Jun 30 12:52:04 2022] GET /identity/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:06.122764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19bfb-3911-450b-a339-6dee7cecaf27 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:06.123326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19bfb-3911-450b-a339-6dee7cecaf27 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:06.123326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf19bfb-3911-450b-a339-6dee7cecaf27 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:06.130027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:06.139147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:06.139542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:06.139807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:06.167208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:06.168763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:06.179458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 12:52:06.202187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19bfb-3911-450b-a339-6dee7cecaf27 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:06.218243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf19bfb-3911-450b-a339-6dee7cecaf27 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:06.224228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:06.224867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:06.225150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:06.225432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:06.234146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:06.235960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1003757-af1f-4112-8c9c-db1501a68098 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:06.245359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Thu Jun 30 12:52:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:52:06.251771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:06.264225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f871-378c-4bfd-96ee-b1fa94a216c7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:06.270301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:06.271214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:06.271501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:06.271778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:06.273362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:06.275265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc07380-09d4-43b3-a54f-4c41b0b86929 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:06.276920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Thu Jun 30 12:52:06 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:08.340043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832059e-fc2e-4a03-9891-657046f2e149 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.340606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832059e-fc2e-4a03-9891-657046f2e149 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.340606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832059e-fc2e-4a03-9891-657046f2e149 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.341950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:08.347284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.347710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.347851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.364097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e05c-67bb-4e60-98cd-53c57a2dfa97 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.364385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e05c-67bb-4e60-98cd-53c57a2dfa97 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.364781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9e05c-67bb-4e60-98cd-53c57a2dfa97 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.365814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:08.366933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:08.367665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:08.374198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.374652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.375075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.415355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG passlib.handlers.bcrypt [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80101) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 12:52:08.415788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG passlib.handlers.bcrypt [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80101) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 12:52:08.452177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:08.460561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:08.462173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:08.475706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.500168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:08.562279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:08.577973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed714c9c-183c-4331-8088-2cad912251a0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:08.584619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.611535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:08.612352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.612660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.612931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.625220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:08.636718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:08.637804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:08.638814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:08.643931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.686184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6b0cb-1161-45f7-9fbf-8751873fa37f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:08.689421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:08.724151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:08.729214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:08.730198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.730571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.730919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.742777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa789a17-0e1b-43e3-a2c3-0f1d0aeed140 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:08.775999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:08.776666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.776939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.777213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.777971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:08.778271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:08.779705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:08.791530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:08.797115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.798233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:08.799802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:08.894917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec4eb1-7875-44b3-9909-96fa174941ab None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:08.896168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:08.901951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.905067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:08.929594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:08.930497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:08.930829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:08.931101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:08.932298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:08.933766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:08.935508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:08.940993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:08.955732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:09.000843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d7dff1-a721-4105-9474-2e9ce3981924 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:09.034865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:09.035662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:09.035983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:09.036253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:09.040986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57a20f-998c-4e84-bbd2-95bb3d70d902 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:09.055109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:09.056610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:09.075627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:09.076437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:09.076723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:09.077001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:09.078213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:52:09.079172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:09.080041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:09.081017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:52:09.081017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:52:09.081017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:52:09.081017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:09.081237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:09.081312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:09.081796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:09.086241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:09.087183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:09.144288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:09.165994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:09.185870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebc571-9535-41f6-9635-5d61b6925209 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:09.189194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055f31f-6b16-4c9b-8309-8618f22e2fe1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:09.222599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:09.223001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:09.223259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:09.223543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:09.223636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:09.223824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:09.223923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:09.224194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:09.224658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.domain.id=default, user.id=b45b25b709bf447ca67a413397143f8d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:09.224925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:09.225027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:09.225259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:09.226446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df8940-618b-4b8c-b813-62601c598a7b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:09.226677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5918b-cdbd-471f-a4a8-54d922e2f077 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:09.233819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 11/20] 10.222.0.49 () {56 vars in 1322 bytes} [Thu Jun 30 12:52:09 2022] GET /identity/v3/role_assignments?user.id=b45b25b709bf447ca67a413397143f8d&scope.domain.id=default&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:09.245247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 10/21] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:11.419193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccecb6-735b-4351-859f-6ecb2cc010e6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:11.420231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccecb6-735b-4351-859f-6ecb2cc010e6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:11.420817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccecb6-735b-4351-859f-6ecb2cc010e6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:11.422433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 12/22] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:11.428294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:11.428706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:11.428892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:11.455395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:11.456613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:11.544180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 11/23] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:11.584355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccecb6-735b-4351-859f-6ecb2cc010e6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:11.726999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccecb6-735b-4351-859f-6ecb2cc010e6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:11.768450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:11.769322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:11.769606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:11.769879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:11.784334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6edbea49-7158-4667-823e-a7a73f94e549 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:11.792179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:11.792973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:11.793729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 13/24] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:11.800376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:11.832300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b15a-7ed2-49c4-bb85-eb0035f581b2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:11.832631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b15a-7ed2-49c4-bb85-eb0035f581b2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:11.832853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817b15a-7ed2-49c4-bb85-eb0035f581b2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:11.834288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 14/25] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:11.840342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:11.840629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:11.840907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:11.848557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:11.872857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:11.874065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:11.910066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:12.039635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e503d4-0c86-4c13-871b-69bc167951c5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.058580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 15/26] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:12.094402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.113352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.114039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.115718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.116012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.117068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:12.117385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.118831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29e06-f4d0-4e42-890d-1960650a3bd2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.152252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 12/27] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:12.153655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8c94-90f3-4bb3-9439-2eefd8b4fe1b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.153945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8c94-90f3-4bb3-9439-2eefd8b4fe1b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.154225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d8c94-90f3-4bb3-9439-2eefd8b4fe1b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.160690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 13/28] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:12.161859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.162142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.162446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.185645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:12.186871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:12.235844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09419a7d-1bd7-4b41-a1f9-fa567c1c8847 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.260512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 14/29] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:12.266340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.283627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.284462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.284846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.285199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.286205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.291125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.294493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:12.315041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:12.381831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16f3e7-5557-4831-b0a9-82bacd2ba3c9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.421281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.422510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.422932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.423381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.425105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.426752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c209435-14cb-4b2a-b407-c18edd1c3366 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.429093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 15/30] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:12 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:12.431022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdaff9-ff3f-4f91-9917-6e6e184160ea None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.431470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdaff9-ff3f-4f91-9917-6e6e184160ea None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.431915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdaff9-ff3f-4f91-9917-6e6e184160ea None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.433721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 16/31] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:12.438792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdaff9-ff3f-4f91-9917-6e6e184160ea None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.451135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 16/32] 10.222.0.49 () {60 vars in 1131 bytes} [Thu Jun 30 12:52:12 2022] POST /identity/v3/roles => generated 219 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:52:12.452339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.569731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdaff9-ff3f-4f91-9917-6e6e184160ea None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.580567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:12.587660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f555d4-4021-4d77-a0fb-1faf04042815 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.632084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.633256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.633978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.635320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.649857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.651754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.668249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.669080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.669365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.669641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.671024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:52:12.672517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:12.673285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:12.674086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:52:12.674086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:52:12.674086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:52:12.674086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:12.674277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f559-df51-46eb-be2f-e893b303c73a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.674783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:12.675941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.676230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.676506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.676883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 17/34] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:12.680645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.711762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:12.713626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:12.785406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:12.801110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 18/35] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:12.802815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:12.851245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cd86c-8b2c-4435-9ff3-266cd70601a9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.893502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.896526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.897003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.897455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] PATH_INFO: `/v3/domains/default/users/b45b25b709bf447ca67a413397143f8d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.900421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b45b25b709bf447ca67a413397143f8d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.902149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:12.941206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75545c9-0a5a-42a9-bd62-c926d0e18886 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:12.976848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:12.977761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:12.978214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:12.978661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:12.979535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:12.979976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:12.981519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.000132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 19/36] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:13.001189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:13.043444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 18/37] 10.222.0.49 () {58 vars in 1361 bytes} [Thu Jun 30 12:52:12 2022] PUT /identity/v3/domains/default/users/b45b25b709bf447ca67a413397143f8d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:13.044861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:13.067467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:13.170262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5d26e-9da9-4096-84de-0d87f1ceb84c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:13.206808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:13.211420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:13.212597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.212981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.213353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.214386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:13.216199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f670-cbdc-4efa-a3c3-4b3df0ada94c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.239747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306deda-adcc-46b0-867f-eb40922accae None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:13.253639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 19/38] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/projects => generated 301 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:13.255169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d1ab-344e-4383-90fe-13b8d203f281 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.255461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d1ab-344e-4383-90fe-13b8d203f281 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.255742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436d1ab-344e-4383-90fe-13b8d203f281 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.258508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 20/39] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:13.258508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0480a-162d-423d-84f6-332c2a9c4327 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.258508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0480a-162d-423d-84f6-332c2a9c4327 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.258773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0480a-162d-423d-84f6-332c2a9c4327 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.259989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 21/40] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:13.265747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.265747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.265747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.284123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:13.285717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:13.286055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:13.290411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.290411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.290411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.290411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:13.290833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a28fe5f-a298-4d0e-a17e-e120c7e615a0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.314560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 20/41] 10.222.0.49 () {60 vars in 1131 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/roles => generated 225 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:13.316239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.316781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.317131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.355962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:13.357387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:13.366805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:13.370017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 22/42] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:13.395282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:13.472829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 21/43] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:13.497997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:13.527049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d817edf-3f94-4cb2-a862-65dcb499206e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:13.546079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:13.575779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:13.576583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.576870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.577150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.577938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:13.580285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.605196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 23/44] 10.222.0.49 () {60 vars in 1131 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/roles => generated 223 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:13.616776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43069ad7-1baf-4c84-8ff6-2010449981f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:13.656261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:13.657180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.657467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.657742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.659202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:13.660687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7969f7-35e5-4526-929c-05cd80958dce None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.663110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 22/45] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:13.668532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:13.739237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:13.785398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff6ca3-15ee-4a66-9037-c99d3487ece9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:13.819222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:13.819887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:13.820168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:13.820442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:13.821169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:13.822638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acace25-aa87-43f3-8e86-115cb21f650b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:13.852872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 24/46] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:52:13 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:15.168767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71f95-8d1f-49bd-b44a-648395d8b61d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.169318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71f95-8d1f-49bd-b44a-648395d8b61d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.169318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71f95-8d1f-49bd-b44a-648395d8b61d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.170855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 23/47] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:15.176406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.176813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.177025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.194044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:15.195248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:15.271300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 25/48] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:15.294502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d71f95-8d1f-49bd-b44a-648395d8b61d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:15.402788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d71f95-8d1f-49bd-b44a-648395d8b61d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:15.413844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:15.437491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:15.438261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.438569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.438844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.448965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:15.456937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:15.457801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:15.458587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 24/49] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:15.464570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:15.510994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:15.567227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f75d82-4cb0-4858-aa52-1ad21de7b3b2 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:15.603980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:15.604637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.604916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.605188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.605931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:15.606234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:15.607641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:15.616218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 26/50] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:15.621984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:15.724016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081567d-fe55-43f3-a0f6-3105eb793750 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:15.759043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:15.759909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.760188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.760459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.761659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:15.763154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce95c45-cc55-4d45-a175-62a4ca7ea036 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:15.764804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 25/51] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:15 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:15.770470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:15.851463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80144aae-3b1b-4f4b-82ca-591895de8f08 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.851764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80144aae-3b1b-4f4b-82ca-591895de8f08 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.852033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80144aae-3b1b-4f4b-82ca-591895de8f08 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.853640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 26/52] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:15.859126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:15.859412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:15.859727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:15.877919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:15.879195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:15.916319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:15.955661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 27/53] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:15.980598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:15.994216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d5afb-3d56-4e73-8a81-ebf3419548d7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:15.996737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:16.028707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:16.029507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:16.029786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:16.030069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:16.031250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:16.032929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:16.034749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 27/54] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:16.040445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:16.079811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ba830-424b-43dd-aa65-a2ce69b15e23 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:16.113313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:16.114112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:16.114470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:16.114727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:16.115927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:16.117407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:16.119048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 28/55] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:16.124630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:16.137178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999cacc4-01c7-4b04-8b7f-9960bc0ef31f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:16.159595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:16.171283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:16.171948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:16.172292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:16.172567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:16.173515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.domain.id=default, user.id=b45b25b709bf447ca67a413397143f8d) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:16.173832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:16.175309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68dbd62-99f0-451f-b83b-4aa24a801a72 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:16.183605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 28/56] 10.222.0.49 () {56 vars in 1322 bytes} [Thu Jun 30 12:52:16 2022] GET /identity/v3/role_assignments?user.id=b45b25b709bf447ca67a413397143f8d&scope.domain.id=default&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:16.226903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d328dd6-8eb5-412c-bd88-422e947bafb1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:16.265450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:16.266179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:16.266500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:16.266768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:16.267511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:16.269058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dac8b-b3f5-4440-a837-a19b7145d13e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:16.299855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 29/57] 10.222.0.49 () {60 vars in 1140 bytes} [Thu Jun 30 12:52:16 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:52:18.306447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd072-f84b-4a32-841a-80555874cbb6 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:18.306954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd072-f84b-4a32-841a-80555874cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:18.306954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd072-f84b-4a32-841a-80555874cbb6 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:18.308398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 29/58] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:18.315068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:18.315747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:18.316041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:18.339308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:18.340523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:18.413892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 30/59] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:18.441409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fd072-f84b-4a32-841a-80555874cbb6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:18.484956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:18.566676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fd072-f84b-4a32-841a-80555874cbb6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:18.603322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:18.604108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:18.604395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:18.604673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:18.605912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:18.607406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a63d6-d3c3-4175-aa8e-c1de09194a82 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:18.609018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 30/60] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:18.617394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:18.636886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:18.720451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244f10-51cd-4f82-bf49-d039573bcfd2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:18.759748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:18.760416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:18.760706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:18.760984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:18.761700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:18.763207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8fcf4-30fb-4fe4-bc83-e5976dc0e27e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:18.793211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 31/61] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:18 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:52:21.921489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ade0f-9a31-4e18-885f-3c23bccab1f5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:21.922083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ade0f-9a31-4e18-885f-3c23bccab1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:21.922230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6ade0f-9a31-4e18-885f-3c23bccab1f5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:21.924205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 31/62] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:21.929438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2f18-0fa5-4456-897b-e0c91b5ed1aa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:21.930241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2f18-0fa5-4456-897b-e0c91b5ed1aa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:21.930725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec2f18-0fa5-4456-897b-e0c91b5ed1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:21.955412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-60ec2f18-0fa5-4456-897b-e0c91b5ed1aa None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:21.956891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-60ec2f18-0fa5-4456-897b-e0c91b5ed1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:22.036272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 32/63] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:22.074482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ade0f-9a31-4e18-885f-3c23bccab1f5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.095483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f61b9-6e90-4dd4-8a36-f85e903e26f9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.095783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f61b9-6e90-4dd4-8a36-f85e903e26f9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.096053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f61b9-6e90-4dd4-8a36-f85e903e26f9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.097453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 33/64] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:22.102354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7cf3-6d5e-4b48-bb46-8a7da26358e3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.102666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7cf3-6d5e-4b48-bb46-8a7da26358e3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.102945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7cf3-6d5e-4b48-bb46-8a7da26358e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.114444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.136732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bf0f7cf3-6d5e-4b48-bb46-8a7da26358e3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:22.139617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf0f7cf3-6d5e-4b48-bb46-8a7da26358e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:22.213223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6ade0f-9a31-4e18-885f-3c23bccab1f5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.227026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.253390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.254376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.257819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.258236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.277105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:22.286857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:22.287712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:22.288460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:22.289014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 32/65] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:22.294605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.300288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 34/66] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:22.325309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9e77-5e9d-43f7-8ed8-593c2d2697fb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.325600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9e77-5e9d-43f7-8ed8-593c2d2697fb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.325886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db9e77-5e9d-43f7-8ed8-593c2d2697fb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.327292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 35/67] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:22.332577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.332860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.333142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.350821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:22.352366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:22.411449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b040f-f4ff-4b83-b3ef-f9a413c80ab6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.429576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 36/68] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:22.430850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.449543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.450327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.450644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.450926 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.452010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:22.452323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:22.453943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130244a-102b-4dea-bf62-bad66bb0ca6c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:22.463356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 33/69] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:22.464615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afa6d-6e12-417f-a64c-c631e6245a8b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.464909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afa6d-6e12-417f-a64c-c631e6245a8b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.465187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afa6d-6e12-417f-a64c-c631e6245a8b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.466723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 34/70] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:22.467118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afa6d-6e12-417f-a64c-c631e6245a8b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.500725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.503674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.545156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614bab06-f37e-4818-ae04-690de22c5787 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.580673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.580897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afa6d-6e12-417f-a64c-c631e6245a8b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.581483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.581768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.582042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.592341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:22.600084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:22.600931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:22.601656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 37/71] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:22.602376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.619844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.620640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.620921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.621192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.631233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e8122d08-bb80-4a4b-96fa-ba9261cf1564 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:22.638848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:22.639867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:22.640637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 35/72] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:22.641245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9da27f-5a99-40d2-a9c7-5383c9cab141 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.641756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9da27f-5a99-40d2-a9c7-5383c9cab141 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.641849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9da27f-5a99-40d2-a9c7-5383c9cab141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.659010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-dc9da27f-5a99-40d2-a9c7-5383c9cab141 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:22.660217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dc9da27f-5a99-40d2-a9c7-5383c9cab141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:22.704290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921b4c-8e19-46f2-9c1c-0aaa990eeaf7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.725094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 36/73] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:22.726368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216da5-0226-449e-8ff0-9821bbec7f45 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.726647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216da5-0226-449e-8ff0-9821bbec7f45 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.726927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216da5-0226-449e-8ff0-9821bbec7f45 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.728344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 37/74] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:22.728905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216da5-0226-449e-8ff0-9821bbec7f45 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.731510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.743147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.744027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.744316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.744590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.759840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:22.761344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:22.761745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.763119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 38/75] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:22.763904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.843656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216da5-0226-449e-8ff0-9821bbec7f45 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.871752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7589f9-1a4f-4c99-807c-0a8f753b6038 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:22.877767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.878460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.878745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.879018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.879768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:22.880106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:22.881529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146155b-0a5a-40fa-a11c-aa05429ba338 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:22.889728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 38/76] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:22.891195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.891485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.891760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.907897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:22.909093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:22.910100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:22.910834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:22.911265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:22.911542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:22.912307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:22.912603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:22.914002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:22.926991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 39/77] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:22.926991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.972611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:22.980792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 39/78] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:22.981563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:22.991113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.029524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73263038-a7e2-4936-81db-9c42fe49a33a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.061773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.062861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.063255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.063534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.073442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:23.081773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:23.082806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:23.089212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:23.089920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f919f4c-5618-4f39-b227-d1d48c8dfab7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.090001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 40/79] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:23.090001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.116163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.116809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.117094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.117366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.185128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.186477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc786ef-49a8-4cf9-8598-190aeb0ae779 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.186653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.190013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.196657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.220810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.221690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.221974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.222251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.223566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.225178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.226864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 41/80] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:23.231166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.285821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 40/81] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:23.286709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.333669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aad7f-3250-4435-ba7e-4eab500543c8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.365666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.366579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.366857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.367125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.368321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.369784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.371623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 42/82] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:23.372214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.387402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3107b-a47f-4649-97b6-743e4b947228 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.397398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.415557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.423302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.424105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.424386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.424660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.425864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.427368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.429239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 41/83] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:23.429781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.473518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09f33c-9f06-413a-b252-3bde8fbbdf5d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.508178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.508774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.509059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.509319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.510080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:23.510418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.511834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.519825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 43/84] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:23.520611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.533405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4200c57-08b4-4dde-b16d-6675b692e13a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.565842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.566523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.566803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.567074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.613959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.620707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab518b-867e-4680-b10f-e01aec3977ab None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.622219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.638010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.639520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.655161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.656026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.656350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.657215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.658493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.659967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.661673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:23.665218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.740736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 42/86] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:23.741693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.767942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea136a7-9c23-4c5f-b804-f76f8d12bd85 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.801905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.802581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.802872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.803159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.817995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.840415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785122e2-5c96-4c07-8708-e9c1ef1f288c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:23.841926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:23.870967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.872466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.875700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:23.876364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:23.876644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:23.876917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:23.877641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:23.879105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:23.905955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 43/87] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:23 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:23.906920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:23.990051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 45/88] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:23.990979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.020072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4070a4-8bcb-4772-a845-f6e4ffc20773 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.056665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.062879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.063982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.064447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.064902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.066314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.067841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.069727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 44/89] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.070367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.072829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.113522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23abb27-2625-4e24-9fb5-67615f5c5daa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.154160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.154894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.155190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.155471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.156416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:24.156752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.158539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.165913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 46/90] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:23 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=87f5d4315954429eb4783c57c3909faf => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.166618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.201400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eff0d2-c1e3-4e1a-b2d1-e6703aaf1fe1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.254214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.255150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.255479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.255974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.257017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.258607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.260536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 45/91] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.261534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.308118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314cdff-780e-4e80-8b32-94a5d06bf7a7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.347123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.348579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.349058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.349524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] PATH_INFO: `/v3/projects/a242b48f4de7458a90955e38f97dae9f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.351162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] RBAC: Authorizing `identity:get_project(project_id=a242b48f4de7458a90955e38f97dae9f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.353240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.353827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.355949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/projects/a242b48f4de7458a90955e38f97dae9f => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.356974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.361981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.392508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f1d5-7edf-4cc7-9a42-1c39ffdf6343 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.430064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.430763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.431045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.431326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.467158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3ce1e-dd59-40ae-b331-7dbfbf9a6f79 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.502849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.503408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.504082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.504346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.504455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.504625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.505569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=b45b25b709bf447ca67a413397143f8d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:24.505888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.507432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.513552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 48/93] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/role_assignments?user.id=b45b25b709bf447ca67a413397143f8d&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.519389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.680470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.696391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:24.755082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 46/94] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:24.772490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:24.836034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1ca64-46a9-4501-aca3-637ca22d8b4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.923905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4831fed-64ac-4113-a0c2-dca9f7d48878 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:24.936695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.937326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.937692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.937978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.943207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=a242b48f4de7458a90955e38f97dae9f, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:24.944676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.945046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.955947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 49/95] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=a242b48f4de7458a90955e38f97dae9f&role.id=87f5d4315954429eb4783c57c3909faf => generated 259 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.979256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:24.980141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:24.980426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:24.980707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:24.981969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:24.983472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e366-48f3-448d-8665-e253ea1c9bfc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:24.985681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 47/96] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:24.991521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:25.142212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2457002-36eb-4c0b-a5d0-22191a65eb58 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:25.198909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:25.211215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:25.211923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:25.212568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:25.212852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:25.213714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:25.214039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:25.215594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03bf3-2276-4033-af6a-824a656b773a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:25.222070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 50/97] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:24 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:26.749339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11ee8-2a76-4218-b1d7-81782c4ed52c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:26.749893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11ee8-2a76-4218-b1d7-81782c4ed52c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:26.750012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd11ee8-2a76-4218-b1d7-81782c4ed52c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:26.751520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 48/98] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:26.757608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:26.757608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:26.757608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:26.778136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:26.779661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:26.876515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 51/99] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:26.924119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd11ee8-2a76-4218-b1d7-81782c4ed52c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:26.971420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:27.060154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd11ee8-2a76-4218-b1d7-81782c4ed52c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:27.103980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:27.105141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.105638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.106110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.107480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:27.109135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5929ed-8fc3-40a9-b89b-2648056d1d91 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:27.111072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 49/100] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:27.117137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:27.270878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38a571-7447-44ae-a658-1db2396763a1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:27.311489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:27.329406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:27.330097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.330405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.330690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.331441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:27.332924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa035a35-42dd-4635-85c4-62a61f942d26 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:27.363764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add2213-69fb-4a58-8c99-db6c18e9ff4e None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.364492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add2213-69fb-4a58-8c99-db6c18e9ff4e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.365041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add2213-69fb-4a58-8c99-db6c18e9ff4e None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.370964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 50/101] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:27.374563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea329-10e8-44ba-8f51-0817168246ca None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.375195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea329-10e8-44ba-8f51-0817168246ca None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.375661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ea329-10e8-44ba-8f51-0817168246ca None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.377192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 51/102] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:27.379409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.380648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 52/103] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:52:27 2022] POST /identity/v3/projects => generated 302 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:52:27.381193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.382770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.393258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2727-7990-4c6e-8bd7-71d88ec458a8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.393876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2727-7990-4c6e-8bd7-71d88ec458a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.394377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f2727-7990-4c6e-8bd7-71d88ec458a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.424146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:27.425716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:27.426870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ba6f2727-7990-4c6e-8bd7-71d88ec458a8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:27.428284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ba6f2727-7990-4c6e-8bd7-71d88ec458a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:27.542541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 52/104] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:27.548833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 53/105] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:27.577464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:27.581676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be435-0cc1-4e24-a08b-d0392f800262 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.581972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be435-0cc1-4e24-a08b-d0392f800262 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.582247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4be435-0cc1-4e24-a08b-d0392f800262 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.583914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 54/106] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:27.589832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.590117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.590450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.611876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:27.613544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:27.617264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:27.726900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 55/107] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:27.728440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:27.736863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589684f5-9abd-4a9c-a7fd-801e4cd641ab None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:27.774476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:27.793060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:27.794506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.794949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.795447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.808367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:27.817213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:27.818201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:27.818987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:27.821634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 53/108] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:27.823723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:27.857542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f782ea3-96dc-4647-a478-ace735ad36dc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:27.905051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:27.905870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:27.907205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:27.907494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:27.922719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:27.933617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:27.934574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:27.935327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 56/109] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:27.935926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:27.950076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:27.959939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a287d7d-980e-4679-b601-8dbba431ceb0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.005612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.006988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.007699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.008148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.023868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:28.037933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:28.038879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:28.039671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:28.042077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 54/110] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:28.043574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.079551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.088008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e828af-af6d-46d1-b7a8-6e1075d6877b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.134890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.136072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.136491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.137063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.137886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:28.138261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.140001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.151637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 57/111] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.152950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.183670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83142b1-46b8-445a-838d-1a9d578c5640 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.223083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.223759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.224218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.224498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.225255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:28.225770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.227038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5f8c2-68f0-4dc8-810f-8f0b3266407c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.235911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 55/112] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.237013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065c85b-da51-4c3b-ad81-b0a542bea5ea None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.237302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065c85b-da51-4c3b-ad81-b0a542bea5ea None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.237577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065c85b-da51-4c3b-ad81-b0a542bea5ea None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.243236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 56/113] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:28.247917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065c85b-da51-4c3b-ad81-b0a542bea5ea None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.263592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.278348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bec50-fae6-45b4-bd80-172517835fbe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.314043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.315091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.315588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.316044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.316990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:28.317472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.319081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.328327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 58/114] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.329519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.361080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.362351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065c85b-da51-4c3b-ad81-b0a542bea5ea None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.405605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.407525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.407994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.408742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.410604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.412831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006b471-fc50-4af5-a074-8c6f770d7a2c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.415245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 57/115] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.417369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.417795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.418296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.439355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:28.441147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:28.450418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d33dc-1f4f-49f4-a451-f021eb4b4f7c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.488177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.489523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.489983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.490548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] PATH_INFO: `/v3/users/2f853bb33ace454a9916ed839360e68f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.505599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] RBAC: Authorizing `identity:get_user(user_id=2f853bb33ace454a9916ed839360e68f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.507383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b048d9-3775-4f52-a761-f360de6ff49c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.509266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 59/116] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/users/2f853bb33ace454a9916ed839360e68f => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.509368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 58/117] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:28.510306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.510872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aea22-36c6-4126-bc73-c4cdb3ad0b62 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.511224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aea22-36c6-4126-bc73-c4cdb3ad0b62 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.511569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3aea22-36c6-4126-bc73-c4cdb3ad0b62 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.512869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 60/118] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:28.513625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aea22-36c6-4126-bc73-c4cdb3ad0b62 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.538851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.605667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.629546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3aea22-36c6-4126-bc73-c4cdb3ad0b62 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.631758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddc2c5-cbd1-49fc-9776-bc61b5b14e39 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.666044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.666871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.667150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.667427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.668612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.670074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47dfc4-7c9b-4096-bfdf-8ac99995df56 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.671386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.676651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 61/119] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.676794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.676870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.676936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.676999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.677061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.677134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.677263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.678767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.680452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 59/120] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:28.681187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.696399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:28.697585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:28.763789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.768737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 62/121] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:28.769805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:28.797138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443276a3-3cda-44e1-9542-7475c67e9918 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.832100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.833097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.833573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.834039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.854915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:28.879458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00b7cc-8276-48ae-a1d5-681fe2bf4d78 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:28.906438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:28.908468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:28.914469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:28.915303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:28.915588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:28.915862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:28.927485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:28.935654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:28.936468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:28.937201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 63/122] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:28.938005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.002713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.028977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 60/123] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:29.028977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.052252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8233bfe-4faa-4526-a4f8-1a57ee575ed8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.087743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.088648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.088929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.089208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/63b8e8615a52438eba7871052809289f/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.091237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, user_id=63b8e8615a52438eba7871052809289f, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.092720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.097444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.122655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 64/124] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:28 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/63b8e8615a52438eba7871052809289f/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:29.123941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.132770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be729d-5e5f-4829-9de1-c67e5d8b637a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.174130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.177188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.177484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.177860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.179217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.180855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.183542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 61/125] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.184166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.240319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.255225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74659f-9922-41c2-8d4e-cd490bc8a377 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.294067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e7400-5082-41e9-8b7d-205403e6a846 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.298203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.299060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.299922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.299922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.316101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:29.323842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:29.324702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:29.325585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 65/126] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:29.326309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.342185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.343147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.343433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.344893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.345943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:29.346259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.348365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.361046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 62/127] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.361943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.392649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.452639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ec10b-0453-4122-ab63-46fb30712c4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.493862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.494281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9baa3-ecc2-4f88-b1d5-c730e81f72b5 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.495038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.495415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.495961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.498053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.499884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.501728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 66/128] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.506590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.514626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.537776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.538771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.539105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.539384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/b45b25b709bf447ca67a413397143f8d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.541530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, user_id=b45b25b709bf447ca67a413397143f8d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.543015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.559860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 63/129] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:29 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/b45b25b709bf447ca67a413397143f8d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:29.560760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.653098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddea84-4cfe-49fc-bf60-5e6ace697cc9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.675708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6842b6c-a669-4a71-8753-fd5511fcfc67 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.697757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.715776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.716581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.716861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.717135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.718389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.719911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.721907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 64/130] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.722531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.724052 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.724754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.725040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.725320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.726100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:29.726440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.727862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.746385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 67/131] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.747274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.769554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:29.835181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e3c8c-eab3-4c1c-a92c-abba0f12fe74 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:29.871679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:29.872361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:29.872638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:29.872912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:29.873766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=2f853bb33ace454a9916ed839360e68f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:29.874078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:29.875803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:29.882404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 65/132] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/role_assignments?user.id=2f853bb33ace454a9916ed839360e68f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:29.883221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:29.941437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2191cc-712f-4e57-a3ec-8d080a573c91 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.003678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.003678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.003678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.003678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.007146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.008736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa544e-03e6-4219-bfd0-bad36169a37c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.010603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 68/133] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:30.011958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cea93-076f-43b2-99bc-b7b4d4d5066e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.012248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cea93-076f-43b2-99bc-b7b4d4d5066e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.012525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cea93-076f-43b2-99bc-b7b4d4d5066e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.014024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 69/134] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:30.017538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cea93-076f-43b2-99bc-b7b4d4d5066e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:30.031326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fa533-d5dd-4be1-800e-a1d3a45be935 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.035423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:30.084088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.084988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.085272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.085550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.090111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.091635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338053b-deb5-47e5-9a1d-a9dc55942498 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.093708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 66/135] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:29 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:30.094782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.095604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.095604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.110712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:30.117809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:30.119532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:30.189907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:30.191318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:30.309320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9cd67-de25-4fd0-8564-a5f1951316de None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.334850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:30.339025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cea93-076f-43b2-99bc-b7b4d4d5066e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.348333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.349590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.350077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.350588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] PATH_INFO: `/v3/projects/a242b48f4de7458a90955e38f97dae9f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.352064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] RBAC: Authorizing `identity:get_project(project_id=a242b48f4de7458a90955e38f97dae9f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.353850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.358824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 68/137] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:30 2022] GET /identity/v3/projects/a242b48f4de7458a90955e38f97dae9f => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:30.358824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:30.405577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.412505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.413126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.413599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/63b8e8615a52438eba7871052809289f/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.416354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, user_id=63b8e8615a52438eba7871052809289f, role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.418044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.449141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 70/138] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:30 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/63b8e8615a52438eba7871052809289f/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:30.455726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:30.494997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da6b51-e47c-42c1-bf65-8d41b5c51bcc None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.580666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.581414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.581703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.581980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.587477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.588992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.591058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 69/139] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:30.599398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:30.669088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:30.673716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e628d2ec-b2a5-414d-92a4-b660c59d0a26 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.700879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:30.738687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5712a-6074-47db-b437-75900777cf22 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:30.740710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.741638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.741920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.742206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] PATH_INFO: `/v3/projects/a242b48f4de7458a90955e38f97dae9f/users/63b8e8615a52438eba7871052809289f/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.745482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] RBAC: Authorizing `identity:create_grant(project_id=a242b48f4de7458a90955e38f97dae9f, user_id=63b8e8615a52438eba7871052809289f, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.747078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa1ed4-2cbb-45d5-a63a-8cee50f797aa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.777518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 71/140] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:30 2022] PUT /identity/v3/projects/a242b48f4de7458a90955e38f97dae9f/users/63b8e8615a52438eba7871052809289f/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:30.778099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:30.779021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:30.779579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:30.780063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:30.781041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:30.782861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451339db-0ffa-4347-99fa-42fa06b09c2c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:30.806897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 70/141] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:30 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:31.640164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c93e7-6513-4f12-bb6e-45fa89661e19 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:31.640805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c93e7-6513-4f12-bb6e-45fa89661e19 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:31.640805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46c93e7-6513-4f12-bb6e-45fa89661e19 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:31.642278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 72/142] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:31.647798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:31.648255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:31.648342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:31.670536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:31.671837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:31.762872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 71/143] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:31.807441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c93e7-6513-4f12-bb6e-45fa89661e19 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:31.939137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46c93e7-6513-4f12-bb6e-45fa89661e19 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:31.977930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:31.979177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:31.979689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:31.980168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:31.991861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:32.000328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:32.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:32.002105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:32.004613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 73/144] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:32.010935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.042702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:32.114990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46714c4b-10e1-46d4-bb09-c9bda6fd1ef5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:32.149583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:32.150275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.150582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.150856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.151617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:32.151924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:32.153319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:32.162261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 72/145] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:32.167964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.206792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:32.284634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660de97-d543-43e7-8eb7-9b3c1041f5a4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:32.320913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:32.321850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.322134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.322426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.323656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:32.325148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a93287b-d011-4372-ad37-03cb71e5408d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:32.327050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 74/146] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:32.332742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.406573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552e7d-849d-4cce-9f53-eb1bf5dc4207 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.406917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552e7d-849d-4cce-9f53-eb1bf5dc4207 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.407185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93552e7d-849d-4cce-9f53-eb1bf5dc4207 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.408728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 75/147] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:32.414368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.414658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.414934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.443568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:32.444777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:32.466776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:32.471177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd8497-a0ed-4f4a-a6ec-43eed99a0f43 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:32.512889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:32.513903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.514201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.514600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.594624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:32.596152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:32.679765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 76/148] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:32.690730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:32.702201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 73/149] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:32.708234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.717865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.815406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:32.828252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd715ec6-f93d-40c2-a1d0-743934402ddf None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:32.878626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:32.879441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.879724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.879997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.895341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-475b835e-4b8d-4fcb-abcf-84f1e55a003d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:32.911456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:32.912330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:32.913078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 74/150] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:32.916046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dd2c4-542d-46f1-b51b-591d4c59f165 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.916046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dd2c4-542d-46f1-b51b-591d4c59f165 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.916046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dd2c4-542d-46f1-b51b-591d4c59f165 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.918959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 75/151] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:32.919686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dd2c4-542d-46f1-b51b-591d4c59f165 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:32.947798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee9feb-8c83-41cc-87cb-f1601ab680d5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:32.988090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:32.988884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.989161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.989432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:32.992476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:32.993951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc4d00-91d8-42fc-b11a-be2113d910cb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:32.997231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 77/152] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:32.998472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:32.998754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:32.999024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.005467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:33.049266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:33.050495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:33.131540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dd2c4-542d-46f1-b51b-591d4c59f165 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.198244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 78/153] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:33.201520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.215151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.215823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.216102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.216373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.217129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:33.217430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.219386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.231131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 76/154] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:33.236952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.242748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:33.352461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce6a2-5749-40d6-9b7d-c39dc5638622 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.362310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:33.366849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74c987-b548-41b0-9fba-5f45c78b17e9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.389096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.390059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.390445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.390731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] PATH_INFO: `/v3/users/2f853bb33ace454a9916ed839360e68f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.391986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] RBAC: Authorizing `identity:get_user(user_id=2f853bb33ace454a9916ed839360e68f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.393467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.395153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 77/155] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/users/2f853bb33ace454a9916ed839360e68f => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:33.396053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.425069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.425755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.426033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.426304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.427179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:33.427490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.428956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.440144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 79/156] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=87f5d4315954429eb4783c57c3909faf => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:33.440981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.534286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8247b-a93f-4a80-a51c-82ed974279ec None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.585045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.586047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.589917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.590296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.594816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:33.602631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aaeb4-db8e-472b-8b02-ef6a684460c4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.607980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:33.620303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:33.621407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:33.622234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 78/157] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:33.636604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.693131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:33.695061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.696023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.696304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.696578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.697841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.699420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.701377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 80/158] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:33.709310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.790283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8daaf-72f2-4ad8-97f4-a58797be769f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.840192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.840928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.841215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.841541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.842305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:33.846815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.848275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.861891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 79/159] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:33.862932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:33.869204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd2ce6-a2be-4130-b022-61bb53a06120 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:33.926602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:33.927522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:33.927805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:33.930627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/2f853bb33ace454a9916ed839360e68f/roles/0e63f2bb452f4c18bba89f1c42fde457` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:33.935721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, user_id=2f853bb33ace454a9916ed839360e68f, role_id=0e63f2bb452f4c18bba89f1c42fde457)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:33.937289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:33.968027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 81/160] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:33 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/users/2f853bb33ace454a9916ed839360e68f/roles/0e63f2bb452f4c18bba89f1c42fde457 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:34.068499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:34.072376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595bf8e-8d18-46e4-a1c4-92e44d041e75 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:34.117745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:34.118659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.118943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.119218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] PATH_INFO: `/v3/users/b45b25b709bf447ca67a413397143f8d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.123050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] RBAC: Authorizing `identity:get_user(user_id=b45b25b709bf447ca67a413397143f8d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:34.124570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1311b-c359-4b8b-94ed-96b0a1c012cf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:34.127274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 80/161] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:33 2022] GET /identity/v3/users/b45b25b709bf447ca67a413397143f8d => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:34.132726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:34.170712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:34.275448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118a3-29ab-411f-9d4e-87a6a2a16727 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.275735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118a3-29ab-411f-9d4e-87a6a2a16727 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.276020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118a3-29ab-411f-9d4e-87a6a2a16727 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.277427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 81/162] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:34.289342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08378a8-8338-46eb-8519-41941a68dbcf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:34.295870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49e6f-57ec-48b4-a98e-080f2ba3a3e9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.296165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49e6f-57ec-48b4-a98e-080f2ba3a3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.296448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea49e6f-57ec-48b4-a98e-080f2ba3a3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.332960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:34.333602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.333883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.334155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.343482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9ea49e6f-57ec-48b4-a98e-080f2ba3a3e9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:34.345378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9ea49e6f-57ec-48b4-a98e-080f2ba3a3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:34.417815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:34.419354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:34.462941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:34.530284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 82/163] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:34.537023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:34.635876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 82/164] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:34.637081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c8ac-0832-4ccb-9614-339f2922f8a3 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.637366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c8ac-0832-4ccb-9614-339f2922f8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.637643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c8ac-0832-4ccb-9614-339f2922f8a3 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.639195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 83/165] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:34.645145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37206e28-ee6d-4f65-8ff8-9cdb847c5527 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:34.647705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.647991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.648267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.678679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:34.682148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:34.683180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.683652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.683900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.685338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:34.687037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:34.689015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 83/166] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:34 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:34.692610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:34.703394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:34.704629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:34.805571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c742f-5b42-45b3-9d8b-7fee58485eb8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:34.843604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:34.844773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:34.845199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:34.845643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:34.857602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:34.866795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:34.867898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:34.869003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:34.871264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 84/167] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:34.872244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:34.938822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:34.940672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 84/168] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:34.941631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:34.984379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7ca66-8c2b-4434-bb2d-ab0fb51c6052 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.023703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.024831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.025311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.025814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.026965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=b45b25b709bf447ca67a413397143f8d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:35.027507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:35.029219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:35.032095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:35.037838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 85/169] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:34 2022] GET /identity/v3/role_assignments?user.id=b45b25b709bf447ca67a413397143f8d&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:35.043258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:35.177261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f35f4-b3d2-4bb9-93c6-73a485bd24eb None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.237850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.238490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.238774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.239048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.239884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:35.240197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:35.241638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:35.261817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 85/170] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:35.265050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85faf6-c067-447b-bf04-48a9ad23b9fd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.273750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:35.299455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:35.353926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.354767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.355047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.355322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.388657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:35.396534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:35.397371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:35.398219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 86/171] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:35.411617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:35.438709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:35.465171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85635e2-ac54-47ab-9c71-8b242820e5c1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.544022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.544831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.545113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.545388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.546735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:35.548297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:35.550045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 86/172] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:35 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:35.555433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:35.623019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac286b9-f24b-42d7-aa8a-b755e834e1cb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.690106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.690814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.691095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.691365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.692128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:35.692433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:35.693838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39482-0fa7-42f9-a75c-ea5936bbb004 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:35.715705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 87/173] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:35.717068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe457c0-0721-4347-ac5e-323df19eaca8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.717355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe457c0-0721-4347-ac5e-323df19eaca8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.717629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe457c0-0721-4347-ac5e-323df19eaca8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.719169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 88/174] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:35.723463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe457c0-0721-4347-ac5e-323df19eaca8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:35.773410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e35ccd-7a26-4581-9536-f08058c136bd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.799580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:35.848790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:35.851483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:35.851776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:35.852056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:35.887112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:35.945347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe457c0-0721-4347-ac5e-323df19eaca8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:35.948074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:35.950634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07148-c612-4ea3-9f76-228283ce6d33 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.012957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.013849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.014125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.014427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] PATH_INFO: `/v3/users/63b8e8615a52438eba7871052809289f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.017505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] RBAC: Authorizing `identity:get_user(user_id=63b8e8615a52438eba7871052809289f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.018993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358eb5-2b4b-4971-978c-cc2b7fe31af1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.020648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 89/175] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:35 2022] GET /identity/v3/users/63b8e8615a52438eba7871052809289f => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:36.021862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.022167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.022468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.039219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:36.040734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:36.092118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 87/176] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:36.094159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2588d0-2b63-44c7-8e17-38d637aa2a8d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.094761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2588d0-2b63-44c7-8e17-38d637aa2a8d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.095259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2588d0-2b63-44c7-8e17-38d637aa2a8d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.097066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 88/177] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:36.102200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2588d0-2b63-44c7-8e17-38d637aa2a8d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.111794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 90/178] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:36.112484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.158683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.165704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.213046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2588d0-2b63-44c7-8e17-38d637aa2a8d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.231695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf06a8f-360a-485c-8e6c-cf0c9af4843d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.249388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.250116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.250424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.250697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.277412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.278298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.278607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.278877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.280075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.281595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5a0c9-ccf1-4c8d-b531-f1d044421f3f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.283517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 91/179] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:36.284651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.284936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.285207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.302051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:36.303274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:36.328235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.329699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707fec5-44eb-4b73-adbb-2a791b6e6625 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.376034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 92/180] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:36.376882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.395536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.408283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.451591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 89/181] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:36.452995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834232e-b132-48fe-b9f6-eb4eeeb42b33 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.453292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834232e-b132-48fe-b9f6-eb4eeeb42b33 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.453644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834232e-b132-48fe-b9f6-eb4eeeb42b33 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.458370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 90/182] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:36.458370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834232e-b132-48fe-b9f6-eb4eeeb42b33 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.488988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f99b4f-abea-404a-9b95-26d41e624797 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.524922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.525721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.526005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.526283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.536864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:36.544510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:36.545364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:36.546125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 93/183] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:36.546963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.569672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834232e-b132-48fe-b9f6-eb4eeeb42b33 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.605361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.606119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.606499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.606839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.607739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:36.608115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.609817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.616467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 91/184] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:36.617096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.643168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.676060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.690773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d9252-2833-42c1-a4aa-46c722bad317 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.734046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637381-c5f3-4d72-b4b7-0f8bbcd99ba2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.740979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.741815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.742092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.742387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.755716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8db81672-5cb3-46d8-9122-4bdc6bc33599 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:36.764009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:36.764960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:36.765786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 94/185] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:36.767069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faab2-1363-4cce-8714-677466909ce8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.767069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faab2-1363-4cce-8714-677466909ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.767304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17faab2-1363-4cce-8714-677466909ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.779593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.780803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.780803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.781059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] PATH_INFO: `/v3/projects/a242b48f4de7458a90955e38f97dae9f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.782242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] RBAC: Authorizing `identity:get_project(project_id=a242b48f4de7458a90955e38f97dae9f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.783742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.790254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 92/186] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/projects/a242b48f4de7458a90955e38f97dae9f => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:36.790454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a17faab2-1363-4cce-8714-677466909ce8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:36.790560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.790644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a17faab2-1363-4cce-8714-677466909ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:36.863017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 95/187] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:36.864375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb255d4-7fa5-447c-b6aa-c74ae1ff62e8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.864663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb255d4-7fa5-447c-b6aa-c74ae1ff62e8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.864938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb255d4-7fa5-447c-b6aa-c74ae1ff62e8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.866360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 96/188] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:36.866915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb255d4-7fa5-447c-b6aa-c74ae1ff62e8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.888217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.901831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c7b02-1846-42d2-9427-52922ddfe536 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:36.935099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:36.935745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:36.936028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:36.936305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:36.937101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:36.937412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:36.938865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:36.947126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 93/189] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:36.947916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:36.951505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:36.974443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb255d4-7fa5-447c-b6aa-c74ae1ff62e8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.013945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.014660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.014937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.015289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.016216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:37.016523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.017924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15c0e4-481d-46a8-a739-1e78e5c658fe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.025616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 97/190] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:37.031593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.031884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.032162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.047899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:37.049074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:37.050307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8d06-fa79-4053-b01f-44572d44502b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.082025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.082707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.082984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.083255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.084070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=a242b48f4de7458a90955e38f97dae9f, user.id=63b8e8615a52438eba7871052809289f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:37.084380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.085809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.092194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 94/191] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:36 2022] GET /identity/v3/role_assignments?user.id=63b8e8615a52438eba7871052809289f&scope.project.id=a242b48f4de7458a90955e38f97dae9f&role.id=87f5d4315954429eb4783c57c3909faf => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:37.092968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.110366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.135728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 98/192] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:37.143813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.207313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f4d4-d074-4262-b6e7-7d240035ed72 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.233245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.249365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.250212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.250527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.250791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.252039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.253499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.255173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 95/193] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:37.256566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.269977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe4880-e52d-46e4-bbf5-868c4511e3a3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.311826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.312761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.313040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.313311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.328281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.330083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.331695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 99/194] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:37.332462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.351211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.369322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b0e47-1021-45bf-b928-3efbd39228b7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.402453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.403345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.403631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.403908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.405216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.406961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.408624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 96/195] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:37.409373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.432628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a712ab-1ffd-43ed-8a74-37df82e5b2bc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.461190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.466657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.467456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.467738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.468010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.478049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:37.485875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:37.486856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:37.487630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 100/196] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:37.488262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.510895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb71947-399c-4b92-a4a8-ade8bfcd1853 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.545313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.545988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.546256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.546590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.547564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.548999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.569829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 97/197] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:37 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:37.572055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.583827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.593883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e05cf-837d-49c9-8ded-34fb4c6f569e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.632081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.632756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.633040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.633316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.690082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75396f-f852-4332-a774-60e3ce0bc756 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.700339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.713334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.715382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.731983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.732419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.732706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.732981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.808338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.810229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:37.820066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.832025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 101/198] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:37.835290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.937123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 98/199] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:37.937636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:37.938122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:37.950490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac9a61-7666-474f-b83c-6d8f186f81a5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:37.987127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:37.987790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:37.988100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:37.988373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:37.989127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:37.989490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:37.991291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.000651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 102/200] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.001482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.063358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f441f9-6125-4c98-9c74-b12fec9c2ad8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.075249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.105109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.106136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.106464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.106740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.107946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.109560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.111450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 99/201] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:37 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.112214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.119246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51c139-e73e-49b2-b9b5-89d9b608c0ab None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.154727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.156053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.156534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.157003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.158491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.160170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.162104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 103/202] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.163093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.189726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.225897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56518fcc-b5ce-46c6-b548-1ed23250af2c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.262970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.263966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.264244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.264514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] PATH_INFO: `/v3/users/2f853bb33ace454a9916ed839360e68f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.268070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=2f853bb33ace454a9916ed839360e68f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.269544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.271604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 100/203] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/users/2f853bb33ace454a9916ed839360e68f => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.272395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.284922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8acd7-08a2-4b73-aed2-1c65aa49fd56 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.319791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.325741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.326489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.326841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.327043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.327907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=56df01306205469fb27d383cd6658df0) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:38.328229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.329687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.335476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 104/204] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/role_assignments?user.id=56df01306205469fb27d383cd6658df0&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.336285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.386668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc252cf2-830b-4115-9cf5-cdcbb4cc0bc0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.427185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.428167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.428640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.429100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.430144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=56df01306205469fb27d383cd6658df0) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:38.430841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.432557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.437999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.441904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 101/205] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/role_assignments?user.id=56df01306205469fb27d383cd6658df0&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:38.459276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdbf0d-00e5-4e38-b59b-36a19d3c96af None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.498987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.500029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.500487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.500954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.575201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.577016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.670695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.683729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 105/206] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:38.689764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:38.799593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf4f42-a5ea-42c3-8eea-07af294d0781 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:38.807528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:38.835830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:38.837765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:38.838232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:38.838696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:38.840087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:38.841717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add575ae-9954-4829-b2cb-cf24d189fc10 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:38.843684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 102/207] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:38.851650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:39.002585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06ff5a-956d-4e22-a74b-de35485a438d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:39.046249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:39.047647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:39.047937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:39.048221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:39.049073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150, user.id=2f853bb33ace454a9916ed839360e68f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:39.049391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:39.050911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21540cf1-c293-4305-b7c4-1ffdcecf2f82 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:39.061562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 106/208] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:38 2022] GET /identity/v3/role_assignments?user.id=2f853bb33ace454a9916ed839360e68f&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:39.944565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7423c-efba-4cf6-8d72-c040fe546927 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:39.945117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7423c-efba-4cf6-8d72-c040fe546927 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:39.945117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7423c-efba-4cf6-8d72-c040fe546927 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:39.946773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 103/209] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:39.952148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:39.952716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:39.953271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:39.971894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:39.973405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:40.011147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296087b-9595-4d85-bd90-20b0ce27ccc5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.011756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296087b-9595-4d85-bd90-20b0ce27ccc5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.012258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296087b-9595-4d85-bd90-20b0ce27ccc5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.013853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 104/210] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:40.019743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.020252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:40.021083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.022190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.040515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 107/211] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:40.041996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:40.043403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:40.066538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.115652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 105/212] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:40.144996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.186526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07ab2a-63c2-4036-90f8-4d3eb0cea3b6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.225721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.226552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.227024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.227490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.228823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:40.230475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:40.232823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 108/213] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:40.238000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.265479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f05f75-cd11-430d-8595-e56cc9263adc None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.270203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:40.303248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.304307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.304764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.305276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.311083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:40.317825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:40.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:40.328394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:40.329152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:40.332059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 106/214] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:40.338124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.375522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5aa709-b71a-4df8-8293-a99e9e7eac82 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.416213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.416895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.417172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.417441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.418159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:40.419709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:40.454738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 109/215] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:40 2022] POST /identity/v3/users => generated 297 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:52:40.478744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062934d3-b70b-46bb-a44d-97cc7d144d61 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.523700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.524763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.525324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.525798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.526938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:40.527453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:40.529092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:40.540990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 107/216] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:40.548092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.669744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2d01a-bf53-4fe7-98b4-86963eb3834d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.672889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:40.709509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.710443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.710730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.711001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] PATH_INFO: `/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.728266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd43771c8a4ff5bce54d4a56ee4a7c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:40.729854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c889b-faa1-4b00-9eb8-885b80981369 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:40.731739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 110/217] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:40.737927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.743966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d2b75-de48-49e9-a39a-d3217e34ec98 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.744250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d2b75-de48-49e9-a39a-d3217e34ec98 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.744525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d2b75-de48-49e9-a39a-d3217e34ec98 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.746058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 111/218] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:40.751448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df8070-b1f8-45b6-a381-53e7d35a6c60 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.751721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df8070-b1f8-45b6-a381-53e7d35a6c60 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.752000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df8070-b1f8-45b6-a381-53e7d35a6c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.769110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-76df8070-b1f8-45b6-a381-53e7d35a6c60 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:40.770363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-76df8070-b1f8-45b6-a381-53e7d35a6c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:40.774666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:40.857289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 112/219] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:40.878027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eb80-8401-48e1-9dc9-a31d365b5940 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.878354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eb80-8401-48e1-9dc9-a31d365b5940 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.878630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219eb80-8401-48e1-9dc9-a31d365b5940 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:40.880324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 113/220] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:40.880597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d644a-a104-4db0-a9f3-b3f05194d932 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:40.885062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219eb80-8401-48e1-9dc9-a31d365b5940 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:40.921140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:40.921884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:40.922164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:40.922482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.001389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.002897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.018767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.032525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219eb80-8401-48e1-9dc9-a31d365b5940 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.033127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.078808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.079667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.079942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.080211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.094128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2dc8f34e-526f-4c91-bfaf-a31a7c594dca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:41.104582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:41.105570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:41.106352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 114/221] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:41.107296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.109965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.109965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.115527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 108/222] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:41.116532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.136010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:41.137906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:41.222656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 115/223] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:41.223898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.228610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a0c21-d752-490a-aa11-5253c8c78d65 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.267848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.278469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.279152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.279430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.279867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.280626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:41.281071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.282637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.293423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 109/224] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.294620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.328891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.354503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fe868-4b2b-4c96-a376-6e97fd7bead9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.391088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.391999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.392243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.392517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.393739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.395451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.397549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 116/225] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.398607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.404677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afdc65-aa59-45a3-9b5d-72acc4988154 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.444895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.446290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.446607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.446887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.460082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:41.470388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:41.471323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:41.472090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:41.474390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 110/226] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:41.475433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.515066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1d09e-d7db-4d9a-9701-8f4c6578b634 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.528780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.553785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.554732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.555017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.555287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.556491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.557969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.560069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 117/227] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.560906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.580716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.586880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048f3c4-8007-45db-9609-e8d14524c215 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.623582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.624666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.625202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.625649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.626753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=5acd43771c8a4ff5bce54d4a56ee4a7c) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:41.627243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.629137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.636404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 111/228] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/role_assignments?user.id=5acd43771c8a4ff5bce54d4a56ee4a7c&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=87f5d4315954429eb4783c57c3909faf => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.637544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.669436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53b566-e97b-4326-ab46-43d4af47f11d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.709046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.709699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.709980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.710253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.711066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:41.711374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.713133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.722752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 118/229] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.728863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.753045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90606d4-9555-4da7-bdf9-41725db854bd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.769825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.801466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.802597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.802917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.803291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.805128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.807160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.810020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 112/230] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:41.816477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.831540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:41.842647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff68cc6-6107-4e9c-8ac1-405385bc01d3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.881868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.883293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.883784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.884298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.885926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.887780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:41.890016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 119/231] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:41.896036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:41.948893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80536-e35f-4d21-8e85-b19514bd4ca8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:41.988031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:41.988936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:41.989218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:41.989492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/users/56df01306205469fb27d383cd6658df0/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:41.992554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, user_id=56df01306205469fb27d383cd6658df0, role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:41.994160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:42.011140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 113/232] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:41 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/users/56df01306205469fb27d383cd6658df0/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:52:42.054921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e9071-52e5-4e20-ab82-e7504719ad51 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:42.113124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:42.114563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.115049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.115513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.123270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:42.125444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502c460-1b18-4b92-8e90-5c323c664a39 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:42.127547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 120/233] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:41 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:42.133683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:42.200861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:42.282227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971fca-585e-4266-843f-5d181f56bba2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:42.316641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:42.317460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.317738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.318010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/users/56df01306205469fb27d383cd6658df0/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.320024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, user_id=56df01306205469fb27d383cd6658df0, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:42.321482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b773278-1b1b-4a82-8a92-496b0bacc679 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:42.343114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 114/234] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:42 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/users/56df01306205469fb27d383cd6658df0/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:42.913932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9855e-c242-4409-89fd-0a5fca077c6e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.914376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9855e-c242-4409-89fd-0a5fca077c6e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.914551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9855e-c242-4409-89fd-0a5fca077c6e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.916325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 121/235] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:42.921188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333aea-45ed-418e-9414-15243e479132 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.921798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333aea-45ed-418e-9414-15243e479132 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.922072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b333aea-45ed-418e-9414-15243e479132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.942380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aec7a-c0a2-4197-8b17-8d74c6186e4f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.942682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aec7a-c0a2-4197-8b17-8d74c6186e4f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.942935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aec7a-c0a2-4197-8b17-8d74c6186e4f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.945538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 122/236] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:42.951114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:42.951470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:42.951749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:42.961492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4b333aea-45ed-418e-9414-15243e479132 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:42.963075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4b333aea-45ed-418e-9414-15243e479132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:42.980817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:42.981998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:43.000341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:43.086881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 123/237] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 12:52:43.111501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:43.128332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 115/238] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:43.166013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333aea-45ed-418e-9414-15243e479132 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:43.214497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10a7a0-04c9-473b-9036-891eac1ca971 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:43.240094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:43.262372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:43.263204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:43.263484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:43.263773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:43.264935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:43.267825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:43.272234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 124/239] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:43.279933 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:43.309271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:43.372050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b333aea-45ed-418e-9414-15243e479132 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:43.442699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:43.443420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:43.443699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:43.443969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:43.460774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:43.468930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:43.469812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:43.470783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 116/240] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:43.477551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098bd4e5-c75c-40bf-a397-240b2c897186 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:43.485334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:43.518272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:43.519317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:43.519815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:43.520300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:43.521203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:43.522894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:43.554335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 125/241] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:43 2022] POST /identity/v3/users => generated 293 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:43.590059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:43.601512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f544c-7234-447d-9eef-2c9a076bcdb7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:43.642752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:43.643472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:43.643838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:43.644120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:43.644929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:43.645242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:43.646719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:43.657660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 117/242] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:43.663148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:43.777936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:43.816302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0fd992-3d95-4270-aa7a-4cc4cb1e3ff9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:43.859355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:43.860257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:43.860541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:43.860815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] PATH_INFO: `/v3/users/dd53476d105c49c9b877d90f51d1c92a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:43.875958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] RBAC: Authorizing `identity:get_user(user_id=dd53476d105c49c9b877d90f51d1c92a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:43.877444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b571f3e-ea6e-4905-99cc-367bbfcf620a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:43.879234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 126/243] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:43 2022] GET /identity/v3/users/dd53476d105c49c9b877d90f51d1c92a => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:43.884807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.006118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33bcfe-6767-467a-a2f5-a7f121b03d9d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.043948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.044512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.044795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.045072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.071999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:44.122361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:44.123861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:44.155925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d275c-1352-4ee6-95b2-a0985b516a01 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.156867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d275c-1352-4ee6-95b2-a0985b516a01 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.157362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d275c-1352-4ee6-95b2-a0985b516a01 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.158980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 127/244] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:44.170488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.171064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.171720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.195049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:44.196264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:44.234270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 118/245] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:44.242122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.286310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 128/246] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:44.303877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:44.320491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.357549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fd239-5be0-4ad0-aa23-b36b70d3f15c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.396575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:44.400313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.401667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.402292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.402749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.404081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:44.406425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:44.408739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 119/247] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:44.416479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.446580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b185e3-4311-4738-86f3-1061fecf3c10 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.483391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.484536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.485085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.485500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.496486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:44.504909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:44.505793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:44.506601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:44.509265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 129/248] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:44.525410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.527669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057019f8-2c8a-4d19-9132-7e9353d1bbfc None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.549066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:44.564203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.564558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.564842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.565121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.566004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=dd53476d105c49c9b877d90f51d1c92a) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:44.566337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:44.567815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:44.575067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 120/249] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/role_assignments?user.id=dd53476d105c49c9b877d90f51d1c92a&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:44.679144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbd04-9e53-49f4-a78e-ba4511f6ed02 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.697503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:44.715489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.716629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.717139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.717603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.718718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:44.719221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:44.720866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:44.730516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 130/250] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:44.739759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:44.886061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d36a0-0ee0-49e5-bc40-c83b878f53aa None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:44.921802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:44.922724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:44.923006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:44.923282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] PATH_INFO: `/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:44.926091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd43771c8a4ff5bce54d4a56ee4a7c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:44.927710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997cb4-b89e-457d-b348-8c6188114670 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:44.929664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 121/251] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:44.935231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.020096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381e1c1-dfbb-4d79-a424-ead708ae459f None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.020436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381e1c1-dfbb-4d79-a424-ead708ae459f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.020658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381e1c1-dfbb-4d79-a424-ead708ae459f None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.022225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 122/252] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:45.027439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.027722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.027996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.052382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:45.053588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:45.093248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4c96-ee25-4794-9751-e6a581f88e0a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.120697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:45.135591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.136528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.136796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.137072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.138276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:45.139785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534e4267-1b89-46cc-a088-f7c056514109 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:45.141533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 131/253] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:44 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:45.142897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63dc4-e919-4bfd-99c1-acb24f0b6f26 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.143323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63dc4-e919-4bfd-99c1-acb24f0b6f26 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.148152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d63dc4-e919-4bfd-99c1-acb24f0b6f26 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.148278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 123/254] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:45.149757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 132/255] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:45.151193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.165392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.166294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.166756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.185931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:45.187428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:45.270712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:45.290545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 133/256] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:45.291412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.395856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdf618-4ae0-4f6a-aa57-8599954127d7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.426278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf1ff-81f9-41cd-9ce6-f1e272cd4811 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.464939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.466037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.466340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.466630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.478706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:45.487934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:45.488787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:45.489723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 134/257] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:45.490457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.523552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.524485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.524739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.525010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/users/5acd43771c8a4ff5bce54d4a56ee4a7c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.526998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, user_id=5acd43771c8a4ff5bce54d4a56ee4a7c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:45.528465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:45.543302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:45.585253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 124/258] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:45 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/users/5acd43771c8a4ff5bce54d4a56ee4a7c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:52:45.586177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.617819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae8f-ee2d-4aa3-ac39-1800dcd90103 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.666873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.667919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.668202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.668474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.674170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:45.682159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:45.692632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:45.693590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:45.694396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 135/259] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:45.707409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.837037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd672a-34ef-4351-9359-4bebca264e5c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.864991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:45.869443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf20a4b-355e-434d-a0f2-4f98e1133fc6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:45.909924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.911012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.911546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.912018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.913002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:45.913526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:45.915218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:45.926024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 136/260] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:45.931423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:45.953310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:45.953310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:45.953310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:45.953310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:45.954059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:45.954059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:45.955379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:45.985013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 125/261] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:46.004225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.065905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1c252-90a1-46cd-860e-a5d01da16eaa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.105966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.107273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.107770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.108380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.110754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.112446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.114620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 137/262] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:45 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:46.119834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.183222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.194261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f227e4-796d-44c1-ac3a-3dd144c6d084 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.197034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.237484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.238927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.239469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.239944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] PATH_INFO: `/v3/users/56df01306205469fb27d383cd6658df0` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.241460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] RBAC: Authorizing `identity:get_user(user_id=56df01306205469fb27d383cd6658df0)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.243264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.245586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 126/263] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/users/56df01306205469fb27d383cd6658df0 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:46.272611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.285889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c61651-81be-4436-94f0-fc657ce5dcbb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.356419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.357171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.357368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.357644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.396105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d30a0b-3043-4fbc-953d-53d3e3f422dc None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.434178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.435181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.435650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.436199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.444917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.450479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.481596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.491738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.520085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.521908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.609081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 138/264] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:46.613514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.642855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 127/265] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:46.651416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.794446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b484b-77a9-4ce5-94b6-29cb0817aa54 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.800541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.834009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c247-6981-4798-bb91-e5aed5bf8915 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:46.839486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:46.860032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.860941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.861221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.861489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.863028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.864498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.866410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 139/266] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:46.871907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.879872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:46.881386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.881888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.882375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.883766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:46.885413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea08eca-c64e-4844-8da7-eccda128bea4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:46.898383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 128/267] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:46.898383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87f4c-174f-4966-9b62-2a3d41c506c9 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:46.898383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87f4c-174f-4966-9b62-2a3d41c506c9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:46.898383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c87f4c-174f-4966-9b62-2a3d41c506c9 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:46.898988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 129/268] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:46.899945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c87f4c-174f-4966-9b62-2a3d41c506c9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:46.998346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6e197-7773-47ba-9365-35e7b42ad13e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.045303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.045889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.046198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.046574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.047587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=56df01306205469fb27d383cd6658df0) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:47.047883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:47.049542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb36eb3-86db-4ef1-ad27-75c7e1f91851 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:47.066887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 140/269] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/role_assignments?user.id=56df01306205469fb27d383cd6658df0&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:47.068354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.069073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.069073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.093773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c87f4c-174f-4966-9b62-2a3d41c506c9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.123389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:47.124830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:47.149937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.151003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.151478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.151967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.152991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=56df01306205469fb27d383cd6658df0) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:47.153524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:47.155209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fee6f-0a0e-4cd0-a97c-41296b6b008c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:47.170906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 130/270] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:46 2022] GET /identity/v3/role_assignments?user.id=56df01306205469fb27d383cd6658df0&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:47.229841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f65f5-f702-4683-b92c-cfed9d9f8d5c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.230499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f65f5-f702-4683-b92c-cfed9d9f8d5c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.230968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f65f5-f702-4683-b92c-cfed9d9f8d5c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.232888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 131/271] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:47.240348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.243993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.244496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.253484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.254470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.284441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:47.285648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:47.293094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 141/272] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:47.326594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.414477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 132/273] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:47.466901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.474283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e557e5-8688-4718-8781-d1cf5bb0b5d3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.513699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.514561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.514836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.515204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.530717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:47.539175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:47.540033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:47.541253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 142/274] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:47.548342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.580294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c67e3-a046-464f-a8dc-7d9e86e72949 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.616134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.620167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.620978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.621255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.621523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.622846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:47.624826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b2e45-c643-4207-ac94-2d97ecc41126 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:47.627839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 133/275] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:47.629709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22afda-bb33-43bb-8fe5-21695acedb63 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.629991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22afda-bb33-43bb-8fe5-21695acedb63 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.634778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22afda-bb33-43bb-8fe5-21695acedb63 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.636430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 134/276] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:47.637176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22afda-bb33-43bb-8fe5-21695acedb63 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.640286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.679663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edea0e-2994-4eec-b898-0a2d2fd44421 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.716140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.716904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.717189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.717463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.718640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:47.718954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:47.720373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef62a6-29cb-4b91-860f-2d616e44794e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:47.729712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 143/277] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:47.731043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.731332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.731602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.744257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22afda-bb33-43bb-8fe5-21695acedb63 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.748273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:47.749485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:47.779205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:47.779875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:47.780154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:47.780419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:47.781281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:47.782767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:47.814360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 135/278] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:52:47 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:47.814360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.842899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 144/279] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:47.859481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:47.906631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.910991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:47.965377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf77c1c-462d-4b9c-8068-85d31ed7d777 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:47.978198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4521c-8d4f-45fe-8aa9-078a983ef32e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.013869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.014869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.015180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.015457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] PATH_INFO: `/v3/users/7efea12c7f2b4a68be59e6076c3f070b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.023256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.024065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.024859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.025134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.033859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] RBAC: Authorizing `identity:get_user(user_id=7efea12c7f2b4a68be59e6076c3f070b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.035906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.037835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 136/280] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:47 2022] GET /identity/v3/users/7efea12c7f2b4a68be59e6076c3f070b => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:48.038255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:48.048071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:48.050838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:48.051678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:48.052454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:48.053070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 145/281] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:48.060587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.168685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dac810-87ff-4c32-85f8-823d7581a674 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.192725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8d22e-c7b9-4bd9-bc02-d0b0a75ef165 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.194547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:48.196822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:48.212414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.213330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.213613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.214039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.215202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, user.id=7efea12c7f2b4a68be59e6076c3f070b, scope.system=all) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:48.215527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.216991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.224664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 137/282] 10.222.0.49 () {56 vars in 1308 bytes} [Thu Jun 30 12:52:48 2022] GET /identity/v3/role_assignments?user.id=7efea12c7f2b4a68be59e6076c3f070b&scope.system=all&role.id=87f5d4315954429eb4783c57c3909faf => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:48.241935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.242688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.242964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.243237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.244019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:48.244326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.245856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.256121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 146/283] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:48.263350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.382133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1467db1-dd0b-438f-a164-257d659a4b75 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.418680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.419615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.419829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.420095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] PATH_INFO: `/v3/users/dd53476d105c49c9b877d90f51d1c92a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.421389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] RBAC: Authorizing `identity:get_user(user_id=dd53476d105c49c9b877d90f51d1c92a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.422920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.424972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 138/284] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:48 2022] GET /identity/v3/users/dd53476d105c49c9b877d90f51d1c92a => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:48.430387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.553057 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6776f30-b8a0-46aa-b546-0e5b9835c164 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.601237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.602096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.602402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.602667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.607037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.608501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00401c-669d-4f51-8556-cb98f2eec55e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.610283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 147/285] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:48 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:48.616255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.643645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14619f6-4c4d-47bf-8852-c47cacecd6d2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.643992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14619f6-4c4d-47bf-8852-c47cacecd6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.644180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14619f6-4c4d-47bf-8852-c47cacecd6d2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.645747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 148/286] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:52:48.651469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.651745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.652015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.672431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:48.673531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:48.706893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:48.725232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:48.790830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a7ede-0f3f-462f-ba7e-1fe6ad30f00b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:48.799811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 149/287] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:48.825170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:48.826274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:48.826581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:48.826852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/users/dd53476d105c49c9b877d90f51d1c92a/roles/0e63f2bb452f4c18bba89f1c42fde457` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:48.829127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, user_id=dd53476d105c49c9b877d90f51d1c92a, role_id=0e63f2bb452f4c18bba89f1c42fde457)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:48.830646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:48.842353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:48.849317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 139/288] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:52:48 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/users/dd53476d105c49c9b877d90f51d1c92a/roles/0e63f2bb452f4c18bba89f1c42fde457 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:48.960470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747cfb8-a713-4af6-aa14-949df12692fa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:49.005533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:49.007076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:49.007562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:49.008033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:49.028190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:49.037296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:49.038236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:49.043599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:49.049823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 150/289] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:49.055536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:49.172090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c9084-599c-472b-a1bc-e40c235bd6fd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:49.182454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:49.220633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:49.221242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:49.221524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:49.221807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:49.222668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:49.222985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:49.224377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:49.233590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 140/290] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:49.239344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:49.288114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:49.385406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfca2c3-25c8-4a56-b6a7-09baad4008ac None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:49.432854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:49.433737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:49.434016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:49.434287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] PATH_INFO: `/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:49.435528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd43771c8a4ff5bce54d4a56ee4a7c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:49.437097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:49.439196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 151/291] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:49 2022] GET /identity/v3/users/5acd43771c8a4ff5bce54d4a56ee4a7c => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:49.444336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:49.591563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267f50-ed91-426d-889d-a08f2c3ce3ce None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:49.639722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:49.640524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:49.640809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:49.641082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:49.693220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:49.731518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:49.733016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:49.863213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 141/292] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:49.868693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:50.023954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:50.034726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa129c-52c5-4c7e-857e-7d87675ce8d7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:50.084882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:50.085758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.086042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.086330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.087896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:50.089357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b4efe-4bb8-4623-8c04-1ed1b9a9da73 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:50.092135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 152/293] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:49 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:50.096928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:50.197644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:50.241117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f19aa-4e7b-428f-81d7-ba1610db01b1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:50.295301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:50.295965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.296279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.296553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.297535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=5acd43771c8a4ff5bce54d4a56ee4a7c) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:50.297848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:50.299316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715c3a9-74da-4242-b076-3cde5d2de269 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:50.305702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 142/294] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:50 2022] GET /identity/v3/role_assignments?user.id=5acd43771c8a4ff5bce54d4a56ee4a7c&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=87f5d4315954429eb4783c57c3909faf => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:50.534147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43993-6e36-4092-a9d6-c6c0ccef5f41 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.534678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43993-6e36-4092-a9d6-c6c0ccef5f41 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.534678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43993-6e36-4092-a9d6-c6c0ccef5f41 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.536217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 153/295] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:50.541620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2f01-f2bf-4609-8655-9fa437655473 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.542030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2f01-f2bf-4609-8655-9fa437655473 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.542216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f2f01-f2bf-4609-8655-9fa437655473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.561939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-638f2f01-f2bf-4609-8655-9fa437655473 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:50.564530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-638f2f01-f2bf-4609-8655-9fa437655473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:50.644908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 143/296] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:50.660036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb870b-40dc-4253-aa69-40f6e8415725 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.660328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb870b-40dc-4253-aa69-40f6e8415725 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.660608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb870b-40dc-4253-aa69-40f6e8415725 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.662095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 154/297] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:50.667570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.668069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.668536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.672090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb870b-40dc-4253-aa69-40f6e8415725 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:50.686475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:50.688005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:50.762394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:50.773672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 144/298] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:50.799305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:50.813938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb870b-40dc-4253-aa69-40f6e8415725 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:50.854308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:50.855143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.855418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.855683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.866335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d0c75901-62f2-4131-b58d-2c6d96b09c2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:50.873969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:50.875179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:50.876271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 155/299] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:50.877220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858824a8-6b41-4e58-be73-4624121ab00a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.877220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858824a8-6b41-4e58-be73-4624121ab00a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.877220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858824a8-6b41-4e58-be73-4624121ab00a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.882365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 156/300] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:50.882365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858824a8-6b41-4e58-be73-4624121ab00a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:50.903100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:50.943415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da4857-297d-4448-b8de-18a497edc7cd None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:50.978493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:50.979538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:50.979959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:50.980370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:50.990792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858824a8-6b41-4e58-be73-4624121ab00a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:50.991688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-9800b117-2deb-46fb-a87b-8ef7b29c0f6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:51.000236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:51.001305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:51.002280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:51.004544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 145/301] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:52:51.005833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.006118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.006654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.025370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:51.026614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:51.027503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.028175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.028458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.028724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.029470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:51.029770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.031487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.040031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 157/302] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:51.040852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.083789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.103318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 146/303] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:51.115416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.136027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.157924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fc61b-fa22-4a42-b921-51b05e7f3f44 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.192729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.193416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.193696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.193973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.194773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:51.195080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.196487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.208397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 158/304] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:51.209857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.231727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4eb90c-1d4d-49e3-a93d-3a102e5d3685 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.276509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.277638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.277912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.278174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] PATH_INFO: `/v3/users/3c63b4719d3243d2ab8f3d432a299f33` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.292151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] RBAC: Authorizing `identity:get_user(user_id=3c63b4719d3243d2ab8f3d432a299f33)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.293607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.300111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 147/305] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/users/3c63b4719d3243d2ab8f3d432a299f33 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:51.300111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.327690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c97d7-1207-4e2f-85d9-f1ca1f425b55 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.345432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.365673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.366598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.366938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.367324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.368379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.369850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.376324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 159/306] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:51.376324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.388961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.413627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727d822-8293-4980-a7f2-25ca5a171660 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.447940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.448819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.449101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.449379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] PATH_INFO: `/v3/users/7efea12c7f2b4a68be59e6076c3f070b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.450612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] RBAC: Authorizing `identity:get_user(user_id=7efea12c7f2b4a68be59e6076c3f070b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.452061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.454339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 148/307] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/users/7efea12c7f2b4a68be59e6076c3f070b => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:51.455106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.477363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64925395-06b3-4292-a421-e1088347aaf7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.509618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.510266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.510570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.510844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.511691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, user.id=3c63b4719d3243d2ab8f3d432a299f33, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:51.512000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.513450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0283ea05-6057-4bd2-b1e1-f2ce3ffea4cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.518481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 160/308] 10.222.0.49 () {56 vars in 1308 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/role_assignments?user.id=3c63b4719d3243d2ab8f3d432a299f33&scope.system=all&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:51.519652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de824-2b9b-4d63-88f3-0d532716f285 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.519940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de824-2b9b-4d63-88f3-0d532716f285 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.520197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53de824-2b9b-4d63-88f3-0d532716f285 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.521611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 161/309] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:51.522080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53de824-2b9b-4d63-88f3-0d532716f285 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.573808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b7f81-4b75-4bfd-9244-880436a808bd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.587466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.612258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.617575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.618259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.618585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.618860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] PATH_INFO: `/v3/groups` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.619568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.621068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b322547-55f3-4d61-ae80-91fb0010df1b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.646359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 149/310] 10.222.0.49 () {60 vars in 1134 bytes} [Thu Jun 30 12:52:51 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:51.646359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.646359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.646359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.652605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53de824-2b9b-4d63-88f3-0d532716f285 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.671149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:51.672507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:51.713559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.714662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.715297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.715583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] PATH_INFO: `/v3/system/users/7efea12c7f2b4a68be59e6076c3f070b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.718563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7efea12c7f2b4a68be59e6076c3f070b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:51.720200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:51.737794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 162/311] 10.222.0.49 () {58 vars in 1334 bytes} [Thu Jun 30 12:52:51 2022] PUT /identity/v3/system/users/7efea12c7f2b4a68be59e6076c3f070b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:52:51.769506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 150/312] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:51.793635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:51.913642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb4e53-c4a1-4957-b924-2e48ddc7eb70 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:51.948079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:51.948917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:51.949165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:51.949475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:51.957150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:51.961540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:51.970476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:51.971407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:51.972397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 163/313] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:51.977819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:52.081616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:52.085196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68f0d-9b31-40a8-8b4b-d91a359b633b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:52.121639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:52.122707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:52.123185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:52.123678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:52.124805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:52.125493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:52.127112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:52.136693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 151/314] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:52.142416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:52.247158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652354-dd99-4414-827c-b690c05c5bbb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:52.282755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:52.283639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:52.283915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:52.284185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] PATH_INFO: `/v3/users/dd53476d105c49c9b877d90f51d1c92a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:52.285385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] RBAC: Authorizing `identity:get_user(user_id=dd53476d105c49c9b877d90f51d1c92a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:52.286903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:52.288902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 164/315] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:52 2022] GET /identity/v3/users/dd53476d105c49c9b877d90f51d1c92a => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:52.295439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:52.396090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a915d7-a6c6-4dfa-8f04-cb9ce344eafb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:52.430037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:52.431072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:52.431560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:52.432064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:52.463117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:52.501609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:52.503427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:52.605033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 152/316] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:52.611183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:52.679321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:52.714673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522bdb2-d809-4f2b-8bbe-64c8a28441c5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:52.748155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:52.749020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:52.749369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:52.749842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:52.751123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:52.752629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1acf1-ce17-4c4d-ad7e-821041112731 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:52.758425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 165/317] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:52 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:52.760590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:52.821955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:52.863085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ba1a0-e013-400a-84f3-28c359a148a5 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:52.898663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:52.899426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:52.899703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:52.899974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:52.900803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, scope.project.id=ee8de25536f842af955b5f979fd05d27, user.id=dd53476d105c49c9b877d90f51d1c92a) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:52.901110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:52.902587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07b6f0-5ff4-414e-b9e0-f83c54e4b8ea None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:52.914430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 153/318] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:52:52 2022] GET /identity/v3/role_assignments?user.id=dd53476d105c49c9b877d90f51d1c92a&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:53.640520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f37fd-1e96-4d8f-9859-e36aa844b95f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.640947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f37fd-1e96-4d8f-9859-e36aa844b95f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.641118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f37fd-1e96-4d8f-9859-e36aa844b95f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.642675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 166/319] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:53.659139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.659669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.659669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.683386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:53.684601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:53.761861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 154/320] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:53.778293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba461f-b9fb-44a1-83ab-1e7473d7bba8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.778857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba461f-b9fb-44a1-83ab-1e7473d7bba8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.778857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ba461f-b9fb-44a1-83ab-1e7473d7bba8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.780543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 167/321] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:53.783883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:53.785625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.786204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.786525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.809836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:53.811351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:53.844347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:53.890994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 168/322] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:53.900280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b32fe-0d46-4126-90dd-bfa66acedeb9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:53.912934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:53.938746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:53.939544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.939819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.940089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.950130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-05474576-7fc2-4dc9-9a5c-275503599b83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:53.958346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:53.959347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:53.960218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 155/323] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:53.960861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995a63-726a-420a-95fe-8ebcd906d951 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:53.961022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995a63-726a-420a-95fe-8ebcd906d951 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:53.961291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97995a63-726a-420a-95fe-8ebcd906d951 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:53.962956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 156/324] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:53.965631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97995a63-726a-420a-95fe-8ebcd906d951 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:53.979854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.017177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905586b-58d8-454a-bcb3-816a0a51f753 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.049886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.050709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.050991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.051260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.052420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.053876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a26f1-0349-42ad-8a5e-2291064f45ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.055538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 169/325] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:52:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:54.056653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.057228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.057330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.069185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97995a63-726a-420a-95fe-8ebcd906d951 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.073179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:54.074382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:54.084979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.103635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.104281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.104572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.104842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.105585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:54.105888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.107304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.115839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 157/326] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:54.119171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.141346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 170/327] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:54.142124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.207881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.220389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5eeca-98da-4cac-b7a7-c83fb2e5f9c9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.246282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9c352-b836-4ff6-876d-bf85221447bc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.253181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.253843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.254120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.254424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] PATH_INFO: `/v3/groups` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.255121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.256569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.273865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 158/328] 10.222.0.49 () {60 vars in 1134 bytes} [Thu Jun 30 12:52:54 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:52:54.274857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.283087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.284011 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.284288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.284561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] PATH_INFO: `/v3/users/3c63b4719d3243d2ab8f3d432a299f33` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.285864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] RBAC: Authorizing `identity:get_user(user_id=3c63b4719d3243d2ab8f3d432a299f33)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.287425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.290364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 171/329] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:54 2022] GET /identity/v3/users/3c63b4719d3243d2ab8f3d432a299f33 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:54.295134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.327861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.400109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e86189-2c25-4d03-b1e9-94bb5cbb4733 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.418223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b1e3c-dcee-4b15-bd92-3ba7167770ce None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.441569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.442419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.442700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.442974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.453114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.454619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:54.460725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.461855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.462603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.462883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] PATH_INFO: `/v3/system/users/3c63b4719d3243d2ab8f3d432a299f33/roles/0e63f2bb452f4c18bba89f1c42fde457` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:54.463155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:54.463949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:54.464723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 159/330] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:54.467122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3c63b4719d3243d2ab8f3d432a299f33, role_id=0e63f2bb452f4c18bba89f1c42fde457)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.468564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.470445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.479260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 172/331] 10.222.0.49 () {58 vars in 1334 bytes} [Thu Jun 30 12:52:54 2022] PUT /identity/v3/system/users/3c63b4719d3243d2ab8f3d432a299f33/roles/0e63f2bb452f4c18bba89f1c42fde457 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:52:54.594964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:54.599667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43100ee-71b7-4d1e-a2aa-d0f503b42169 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.640220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.641183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.641466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.641750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.642545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:54.642950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.644391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.653883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 160/332] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:54.660374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.770815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91909c5-ace1-49b7-88bd-a303584982ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.807326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.808218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.808498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.808774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] PATH_INFO: `/v3/users/7efea12c7f2b4a68be59e6076c3f070b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.809979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] RBAC: Authorizing `identity:get_user(user_id=7efea12c7f2b4a68be59e6076c3f070b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.811495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfe33c-15b6-40cc-b96f-822e654364ee None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.813573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 173/333] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:54 2022] GET /identity/v3/users/7efea12c7f2b4a68be59e6076c3f070b => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:54.819197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:54.932277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324383a-b8d5-4d5c-8523-43cd28c37517 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:54.970408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:54.971416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:54.971901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:54.972347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:54.973351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] RBAC: Adding query filter params (role.id=87f5d4315954429eb4783c57c3909faf, user.id=7efea12c7f2b4a68be59e6076c3f070b, scope.system=all) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:54.973835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:54.975484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592aec-1608-4cee-a213-5c36e298abb1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:54.989274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 161/334] 10.222.0.49 () {56 vars in 1308 bytes} [Thu Jun 30 12:52:54 2022] GET /identity/v3/role_assignments?user.id=7efea12c7f2b4a68be59e6076c3f070b&scope.system=all&role.id=87f5d4315954429eb4783c57c3909faf => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:56.645043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7d3c-ea05-498b-ad92-9ea0ea2814a0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:56.645580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7d3c-ea05-498b-ad92-9ea0ea2814a0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:56.645580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7d3c-ea05-498b-ad92-9ea0ea2814a0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:56.647262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 174/335] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:56.652880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539dca-a341-4d20-9497-badc967ddd45 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:56.653731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539dca-a341-4d20-9497-badc967ddd45 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:56.654203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40539dca-a341-4d20-9497-badc967ddd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:56.674875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-40539dca-a341-4d20-9497-badc967ddd45 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:56.676467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-40539dca-a341-4d20-9497-badc967ddd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:56.694941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:56.763838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 162/336] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:56.795631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7d3c-ea05-498b-ad92-9ea0ea2814a0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:56.875428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:56.924261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7d3c-ea05-498b-ad92-9ea0ea2814a0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:56.945284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2e3c-d1eb-4b9b-bc76-930c8e0e4509 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:56.945897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2e3c-d1eb-4b9b-bc76-930c8e0e4509 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:56.946393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f2e3c-d1eb-4b9b-bc76-930c8e0e4509 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:56.948212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 163/337] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:56.955667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c7d51-9ad9-4510-98bc-0c1bdad4b6c4 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:56.956252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c7d51-9ad9-4510-98bc-0c1bdad4b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:56.956885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c7d51-9ad9-4510-98bc-0c1bdad4b6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:56.964954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:56.965824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:56.966103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:56.966399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:56.986157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:56.987432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-151c7d51-9ad9-4510-98bc-0c1bdad4b6c4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:56.988850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-151c7d51-9ad9-4510-98bc-0c1bdad4b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:56.994195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:56.995157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:56.995924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:57.002403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 175/338] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:57.003575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.093684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 164/339] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:57.097086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de217-7b08-41ed-b234-804f3c9b4d3a None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.097376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de217-7b08-41ed-b234-804f3c9b4d3a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.097653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15de217-7b08-41ed-b234-804f3c9b4d3a None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.099466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 165/340] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:57.106262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.106262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.106262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.131401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:57.133025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:57.162265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910f5a-a648-4676-923e-a0c12df969cf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.188489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.205927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.206672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.206924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.207196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.210193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:57.210193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.210193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:57.220449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 176/341] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:52:57.221452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.231193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 166/342] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:52:57.232321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.248179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.339512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38925a5b-fc52-4a2f-8565-9c46155cbb19 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.351583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93f095-4864-4f6a-8c32-e3666fc4030f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.376223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.377278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.377734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.378151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.391061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:57.391260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.391781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.392063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.392337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:57.400471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:57.401350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 12:52:57.402152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:57.410192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 177/343] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:57.411424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.463976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.468169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.471338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:57.490702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.527917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63444c88-c632-4e7e-bc39-8a9b3419debc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.569171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.570446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.570947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.571411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.584762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:57.589000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 167/344] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:57.589000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:57.595889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:57.596826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:57.597611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:57.600911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 178/345] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:57.601994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.725177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92e805-e494-4359-8e25-411865b38111 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.734216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a863d51-b85f-419a-99aa-e3fdda97627f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.742340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.742743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.766705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.767361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.767638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.767906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.774415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:57.774415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.774415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:57.781992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 168/346] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:57.781992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.793874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.795150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.795627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.797523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.798878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.800363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:57.802384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 179/347] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:57.803169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.901990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a85f1-cb12-44c2-89b9-0abf686a28f9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.941025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.941693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.943034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.943301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.944060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:57.944366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.945763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f272992-1f69-42f8-81dd-6a51f16ae562 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:57.946119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f928f1f-7aa5-4c70-ad21-a9b44fc103e3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:57.957571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 169/348] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:57.958145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d4b8b-b15e-496b-b194-875072c6c9a4 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.958462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d4b8b-b15e-496b-b194-875072c6c9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.958740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d4b8b-b15e-496b-b194-875072c6c9a4 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.959987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 170/349] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:57.960673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d4b8b-b15e-496b-b194-875072c6c9a4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:57.991873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:57.993483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:57.994198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:57.994675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] PATH_INFO: `/v3/users/3c63b4719d3243d2ab8f3d432a299f33` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:57.996195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] RBAC: Authorizing `identity:get_user(user_id=3c63b4719d3243d2ab8f3d432a299f33)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:57.996300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:57.998626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e16b44-01d6-4876-b804-1383897dbaa6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.001202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 180/350] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/users/3c63b4719d3243d2ab8f3d432a299f33 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.004539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.004842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.005125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.025353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:58.026595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:58.070773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.082374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d4b8b-b15e-496b-b194-875072c6c9a4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.110552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 181/351] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:58.111907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.129043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.129958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.130237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.130614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.137690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.139228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.141037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 171/352] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:52:57 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.141854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.238671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fcece-6053-4645-9b97-747fd3530914 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.256097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.266433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a961f-593d-47b2-be43-28a2fd0635c9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.285256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.286419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.286843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.287490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.309936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.310831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.311204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.311772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.312577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] RBAC: Adding query filter params (role.id=0e63f2bb452f4c18bba89f1c42fde457, user.id=3c63b4719d3243d2ab8f3d432a299f33, scope.system=all) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:58.313028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.314936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.330923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 172/353] 10.222.0.49 () {56 vars in 1308 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/role_assignments?user.id=3c63b4719d3243d2ab8f3d432a299f33&scope.system=all&role.id=0e63f2bb452f4c18bba89f1c42fde457 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.331988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.362817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.377548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.379536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.502337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 182/354] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:58.504034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.586997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fd1d2-5ca8-46a0-a3c5-78156e5b6f04 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.615075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.635384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.636190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.636468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.636739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.637792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4986d7c-f91f-486c-a4d9-ef7cdd229e00 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.648371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:58.656799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:58.657679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:52:58.658477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 173/355] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:58.659127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.680801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.688637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.689430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.689806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.690171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.691293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:58.691915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.693401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2b9db-d785-435f-98c4-0855cf50b0d9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.701429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 183/356] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=87f5d4315954429eb4783c57c3909faf => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.702817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a945c7-0511-491d-81fb-4bc05157bae8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.703230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a945c7-0511-491d-81fb-4bc05157bae8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.703503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a945c7-0511-491d-81fb-4bc05157bae8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.706082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 184/357] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:58.706982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a945c7-0511-491d-81fb-4bc05157bae8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.784986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945c8e5-8965-4f00-ab9e-6d3684c42e70 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.824339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.825611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.826192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.826681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.828248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a945c7-0511-491d-81fb-4bc05157bae8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:58.828865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.830644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b646d6f-2a30-4b5c-ab89-db69e14563ed None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.832656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 174/358] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.834149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.839650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.840191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.858485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:58.860154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:58.872589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:58.873293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:58.873573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:58.873850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:58.875236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:58.875366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:58.876721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:58.878645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:58.896051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 185/359] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:58.897970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.948403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 175/360] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:58.949730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:58.968507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.040695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a10e640-88a8-44d3-b8ad-b95fd24d4510 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.078502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48223d-32fe-45ae-94fe-8910d19846a9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.078709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.079659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.080040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.080410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.082729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.084402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.086551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 186/361] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:59.087755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.133614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.134631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.135268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.135679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.168999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.210538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f162e9-16c6-42bb-896d-f63e09b1b52e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.229495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.231548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.250538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.253224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.254261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.255901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.255901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.272670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:52:59.282560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:52:59.283482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:52:59.284255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:52:59.284842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 187/362] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:52:59.286549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.361462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 176/363] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:59.362352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.420394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf25d002-63e0-4adb-9d34-7b1f3807e110 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.454081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.470071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.470853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.471131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.471405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.472245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:59.472552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.474516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.482401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 188/364] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:59.483325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.500892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f874bf7-ddcb-4eaf-8920-2f6dec438c70 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.518290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.547698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.549186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.549489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.549765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.550570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:52:59.550878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.552296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c78441-254b-4145-9ac0-220bacc46d3c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.562871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 177/365] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:59.564260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696b84-af38-44f2-891a-a56a42c2776e None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.564549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696b84-af38-44f2-891a-a56a42c2776e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.564824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94696b84-af38-44f2-891a-a56a42c2776e None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.571132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 178/366] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:52:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:52:59.571971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696b84-af38-44f2-891a-a56a42c2776e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.633110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162b787-6a60-4249-80ee-93bf0c9d51a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.676609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.677523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.677892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.678176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.679449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.681121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795941-2c14-496f-bab9-e0ce615304fa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.683391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 189/367] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:59.684619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.684911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.685191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.703646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:52:59.704105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94696b84-af38-44f2-891a-a56a42c2776e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.704863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:52:59.742487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.743611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.744082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.744590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.762438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.782214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 190/368] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:52:59.786546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.797704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:52:59.824577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.826397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.916401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767aa00-14d1-4324-a218-5cdd11ce17c4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:52:59.936824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 179/369] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:52:59.938140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:52:59.962782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:52:59.963704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:52:59.963993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:52:59.964259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:52:59.965901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:52:59.967619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:52:59.969561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 191/370] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:52:59.970183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.024627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.030472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.051269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3d2fc-3f77-4007-a69c-6506b949f4a4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.082801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257653b3-8beb-4ae8-95d0-f41416a4c455 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.093731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.094970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.095470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.095932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.107464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:00.117587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:00.118512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:00.119280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:00.123481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 180/371] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:00.123481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.137954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.138976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.139257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.139526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.140808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.142421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.144200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 192/372] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:52:59 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:00.144967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.244109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b753cb3-2423-4b8e-af5a-79f6b463668c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.281239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f05466-da5c-4d27-ba43-1758a4249f16 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.293106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.294204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.294729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.295150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.296180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=ee8de25536f842af955b5f979fd05d27) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:00.296556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.298120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.300357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.305108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 181/373] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=87f5d4315954429eb4783c57c3909faf => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:00.306201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.314943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.332553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.334159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.334646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.335067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.336316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:00.336774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.339811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.351895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 193/374] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:00.358846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.442083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eca2c0-95ca-4a16-bf3e-37132cbaf1df None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.499179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.500341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.500661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.500938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.503259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.504952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.506715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 182/375] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:00.513643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:00.537998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06b9af-b314-4d17-b54b-d2c62a295df9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.586966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.587998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.588296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.588566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.644548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.694092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:00.705545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16e28b-872a-464c-b176-0b519351c229 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:00.718612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.720118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.768609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:00.769295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:00.769586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:00.769859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:00.770771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=ee8de25536f842af955b5f979fd05d27) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:00.771447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:00.773322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:00.788348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 183/376] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:00.852989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 194/377] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:53:00.858949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:01.018358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddcff0-9b4e-4c0c-8b5e-12a7c794751d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:01.069975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:01.077959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:01.079178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.079520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.080159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] PATH_INFO: `/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.086271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] RBAC: Authorizing `identity:get_project(project_id=288f97ff4fc34a4e979d3c926f4a9c7f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:01.088321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffe14b4-afa9-43d5-bb24-2df9cd6217e0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:01.090605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 184/378] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:00 2022] GET /identity/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:01.098133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:01.214786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:01.238854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badd247-a7cd-4103-81a1-99f165705a37 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:01.276596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf082-830f-40dc-a632-983367f56491 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.276887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf082-830f-40dc-a632-983367f56491 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.277150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf082-830f-40dc-a632-983367f56491 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.278984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 185/379] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:01.286153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.286583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.286882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.289344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:01.290225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.290531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.290802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] PATH_INFO: `/v3/groups/fb1654cc07434c188fca5302a8b3dec6` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.315489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] RBAC: Authorizing `identity:get_group(group_id=fb1654cc07434c188fca5302a8b3dec6)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:01.316994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:01.318987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 195/380] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:01 2022] GET /identity/v3/groups/fb1654cc07434c188fca5302a8b3dec6 => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:01.324816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:01.330017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:01.331781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:01.484111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597b657-8ad7-4aa4-8e61-e4418e8fe3c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:01.501057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 186/381] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:01.543953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:01.544638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.544915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.545191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.546870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] RBAC: Adding query filter params (group.id=fb1654cc07434c188fca5302a8b3dec6, role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=288f97ff4fc34a4e979d3c926f4a9c7f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:01.546870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:01.548616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:01.562519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 196/382] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:01 2022] GET /identity/v3/role_assignments?group.id=fb1654cc07434c188fca5302a8b3dec6&scope.project.id=288f97ff4fc34a4e979d3c926f4a9c7f&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:01.570739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:01.755138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9dddc-d262-4a4a-b924-adc328b3f6c0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:01.787292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:01.797262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:01.798113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:01.798431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:01.798708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:01.813348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:01.821295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:01.822210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:01.823066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 187/383] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:01.832986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:01.872764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:02.011026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d770e5-173a-4215-9b35-e985d5c3f12c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.061355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.062086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.062396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.062674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.063429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:02.063734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:02.065116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89c654-0624-4dc1-bb45-ee2251ca0bae None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:02.078685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 197/384] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:53:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:02.084310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:02.111267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ae75-8b19-4bcb-bd09-e1e769b3d6c3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.111996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ae75-8b19-4bcb-bd09-e1e769b3d6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.111996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7ae75-8b19-4bcb-bd09-e1e769b3d6c3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.113445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 198/385] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:02.131525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.131833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.132111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.163428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:02.164671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:02.220439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3231837-02ee-45a9-8966-d980b03e72fb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.274442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.274442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.274442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.274442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.274897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:02.276115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:02.277785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 188/386] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:02 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:02.287303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:02.303167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 199/387] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:02.334932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:02.366705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:02.385968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:02.458016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eecb77f-9034-47d3-a332-29a3e678b1fa None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.508121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6e338-50c6-4c90-947e-6193f1310d6e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.524376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.525693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.526177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.526674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.529042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:02.530937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:02.533233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 189/388] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:02 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:02.538877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:02.600796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.601341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.601625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.601897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.631540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:02.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:02.640614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:02.641380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:02.650915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 200/389] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:02.654962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba7428-d62d-47ce-9bf8-a855c3a97ac6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.656435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:02.684751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:02.698448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.700003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.700601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.701118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/groups/aca615027828472a8ad74f5a7b81cd0b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.703893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, group_id=aca615027828472a8ad74f5a7b81cd0b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:02.705719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:02.729107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 190/390] 10.222.0.49 () {58 vars in 1442 bytes} [Thu Jun 30 12:53:02 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/groups/aca615027828472a8ad74f5a7b81cd0b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:02.775051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:02.824343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181289-122f-44e5-822b-eabdcdd0c58e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:02.870445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:02.871133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:02.871409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:02.871679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:02.872435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:02.872743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:02.874582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:02.886432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 201/391] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:02.892602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:03.038061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95560f-0182-4be7-b742-7fa5166449c8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:03.078281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:03.079170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.079454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.079727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.080968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:03.082466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:03.084471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 191/392] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:02 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:03.090510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:03.220689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb758b5-5752-47bf-9b03-7b14d00432f7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:03.268665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:03.269904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.270424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.270894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.272314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:03.273987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1668d3-0051-4a55-add5-b5632e66c67d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:03.276265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 202/393] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:03 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:03.282246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:03.283349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:03.352549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8a39-b31e-462c-925c-eb5659c1def2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.352896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8a39-b31e-462c-925c-eb5659c1def2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.353271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a8a39-b31e-462c-925c-eb5659c1def2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.355045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 203/394] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:03.361216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.361496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.361770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.367974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:03.399493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:03.400710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:03.432212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f693e-239e-4b7a-8164-42fa9b9514ec None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:03.471535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:03.472427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.472704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.472975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/groups/aca615027828472a8ad74f5a7b81cd0b/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.483974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] RBAC: Authorizing `identity:create_grant(project_id=c2fd0dd0cc73458992236ec1ddeb3150, group_id=aca615027828472a8ad74f5a7b81cd0b, role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:03.485443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:03.514041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 192/395] 10.222.0.49 () {58 vars in 1442 bytes} [Thu Jun 30 12:53:03 2022] PUT /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150/groups/aca615027828472a8ad74f5a7b81cd0b/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:03.529169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 204/396] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:03.564477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:03.690304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3cb42-0599-4ba6-8394-cfbbce79c70d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:03.705299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:03.734877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:03.735999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.736433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.736844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.751576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-9e58858a-b123-4321-8a1d-d286fe0c8016 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:03.759896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:03.761022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:03.761992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:03.764506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 193/397] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:03.770478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:03.874908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30417d-1e35-4ec0-b344-87f4a0a22ad6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:03.899730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:03.902127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702362e9-cdae-414b-bedb-ad1897e69128 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.902898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702362e9-cdae-414b-bedb-ad1897e69128 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.903318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702362e9-cdae-414b-bedb-ad1897e69128 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.905926 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 194/398] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:03.913389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.913948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.914697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.917706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:03.918502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:03.918787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:03.919061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:03.919905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:03.920213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:03.921756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:03.938270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:03.939856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:03.940686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 205/399] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:03.948952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.064645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 195/400] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:04.087839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.160277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406d616-594f-4b07-a9ba-6f5d16c92ef1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.213565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512923e4-de14-4d3b-abbc-133cf1ae836f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.217421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:04.260265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.260265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.260265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.260943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.261985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:04.263636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:04.270344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 206/401] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:03 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:04.270521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.270521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.270521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.270521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.273468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.294717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-3930e802-1fde-4786-8863-2e47e31d0e24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:04.303409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:04.304278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:04.305078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 196/402] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:04.307440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24cd48-fed6-407a-8318-eebba38972e8 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.307440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24cd48-fed6-407a-8318-eebba38972e8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.307440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24cd48-fed6-407a-8318-eebba38972e8 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.308466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 197/403] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:04.313525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24cd48-fed6-407a-8318-eebba38972e8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.382669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:04.441848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ee013-aa0a-4160-8749-80971ffba844 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.454557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24cd48-fed6-407a-8318-eebba38972e8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.488217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.489532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.489945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.490338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.491542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:04.493373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80870317-71b7-4aaf-941a-d3a7982734c4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:04.493713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.494535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.494821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.495099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.495521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 207/404] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:04.495852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:04.496163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:04.496663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.496947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.497227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.497529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:04.511731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 198/405] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:04.512555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.517436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:04.518683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:04.541965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:04.604119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 208/406] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:04.611038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.636240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421ccaf-cd3d-4b0e-952d-a907663373c1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.685495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.686509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.686793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.687061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/groups/aca615027828472a8ad74f5a7b81cd0b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.689207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, group_id=aca615027828472a8ad74f5a7b81cd0b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:04.690673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:04.692477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:04.714912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 199/407] 10.222.0.49 () {58 vars in 1442 bytes} [Thu Jun 30 12:53:04 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/groups/aca615027828472a8ad74f5a7b81cd0b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:04.715756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.778645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968be59-4c31-4b34-88bd-77a396517b57 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.836335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.837632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.837958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.838238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.839616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:04.845337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:04.847422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 209/408] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:04.853473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:04.858882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:04.877931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04502a33-be76-47f4-8576-dbbe5d254fc2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:04.918705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:04.919583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:04.919868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:04.920142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:04.941762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:04.953523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:04.955729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:04.957612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 200/409] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:04.961047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.017844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3ac5-f529-4b2f-9238-4071f9537963 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.058845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:05.091073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.091973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.092252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.092527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.093936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.095420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.096059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978618fb-2b0b-44a7-94c3-3391eec4714f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.097416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:05.103940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.136537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.137213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.137501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.137776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.138632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:05.138939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.141117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.151197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 201/411] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:05.156814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.178096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:05.316530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf571e1-cf95-4190-977a-d22ee4dba552 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.349647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e3f6e-5e97-47d4-8c40-2e519b3c8f16 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.354818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.355781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.356125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.356457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27/groups/aca615027828472a8ad74f5a7b81cd0b/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.359065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] RBAC: Authorizing `identity:create_grant(project_id=ee8de25536f842af955b5f979fd05d27, group_id=aca615027828472a8ad74f5a7b81cd0b, role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.360685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518593f-8405-40c0-bb54-cfb2eb67d40f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.382522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 211/412] 10.222.0.49 () {58 vars in 1442 bytes} [Thu Jun 30 12:53:05 2022] PUT /identity/v3/projects/ee8de25536f842af955b5f979fd05d27/groups/aca615027828472a8ad74f5a7b81cd0b/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:53:05.384876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a762e8-0b5f-4302-b9bd-d0c9487bc222 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.385161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a762e8-0b5f-4302-b9bd-d0c9487bc222 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.385432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a762e8-0b5f-4302-b9bd-d0c9487bc222 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.392854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 212/413] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:05.395511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.396080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.396606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.411653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.412528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.412811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.413088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] PATH_INFO: `/v3/groups/fb1654cc07434c188fca5302a8b3dec6` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.414298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] RBAC: Authorizing `identity:get_group(group_id=fb1654cc07434c188fca5302a8b3dec6)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.415749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.425588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 202/414] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:05 2022] GET /identity/v3/groups/fb1654cc07434c188fca5302a8b3dec6 => generated 229 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:05.425588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:05.430166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:05.534354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:05.565125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:05.570634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 213/415] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:05.596014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.596202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351006a-08e4-40be-9a89-cd7da0fc995b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.645242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.646517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.647040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.647527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] PATH_INFO: `/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.649699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] RBAC: Authorizing `identity:get_project(project_id=288f97ff4fc34a4e979d3c926f4a9c7f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.651521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.653990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 203/416] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:05 2022] GET /identity/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:05.663569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.704716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d344303-1945-4d1c-85be-8efb55649d02 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.748049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.748916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.749203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.749476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.771687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:05.787588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:05.788563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:05.789540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 214/417] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:05.797576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d02eb-7e6d-4a92-be53-505607b86796 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:05.811605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:05.838089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:05.839436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:05.840611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:05.841250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] PATH_INFO: `/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f/groups/fb1654cc07434c188fca5302a8b3dec6/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:05.845905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] RBAC: Authorizing `identity:create_grant(project_id=288f97ff4fc34a4e979d3c926f4a9c7f, group_id=fb1654cc07434c188fca5302a8b3dec6, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:05.849696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:05.871630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 204/418] 10.222.0.49 () {58 vars in 1442 bytes} [Thu Jun 30 12:53:05 2022] PUT /identity/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f/groups/fb1654cc07434c188fca5302a8b3dec6/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:05.886642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:05.957382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fb93-0af1-43b9-874a-1ff11bbe8333 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.008014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.009130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.009610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.010104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.011127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:06.011660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:06.013394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e0af0-a8b1-419a-9668-1b0526aae9a5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:06.026606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 215/419] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:53:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:06.032785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.036428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:06.155782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d414ec8-6238-41b2-94fa-c6877a5e5785 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.190586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.191287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.191569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.191842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.262399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:06.263886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:06.265908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:06.364523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 205/420] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:06.366347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cdb2-a4b0-4885-be3d-0c25d1740c80 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.366795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cdb2-a4b0-4885-be3d-0c25d1740c80 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.366928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cdb2-a4b0-4885-be3d-0c25d1740c80 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.368416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 216/421] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:06.369643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.375696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.375981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.376259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.418595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:06.419853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:06.472412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46287a25-2094-478c-a05e-de0d7c4a0036 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.482766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:06.508380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 217/422] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:06.546595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.546944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.547832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.548111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.548384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.549608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:06.551127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:06.553164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 206/423] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:06.563588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.585183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:06.719422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd37bf-8287-438c-ae52-4ece8bdd31ab None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.746067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226eff2e-1b44-4916-ad98-343e4cc03ff1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.759284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.760114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.760397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.760667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.789474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:06.797626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:06.798597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:06.799356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:06.800496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 218/424] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:06.807275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.871073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.871965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.872246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.872519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.873740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:06.875332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:06.877296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 207/425] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:06.882835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.907230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:06.916941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb523-24fa-4083-9ee1-e6086d5d0f68 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:06.953529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:06.954262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:06.954579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:06.954857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:06.955699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:06.956014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:06.957421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:06.966948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 219/426] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:06.972750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:06.992879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.089355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922a9d-07e6-4d68-830c-86c3bee44937 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.122039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc434ce1-650c-45b7-b9e0-c6f5729a2185 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.131594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.132981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.133266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.133539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.159062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.159721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.160579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.161392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.161676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.162889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:07.163201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.165281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400379cd-0936-48df-bbdd-e71a4cab093a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.174527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 208/427] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=87f5d4315954429eb4783c57c3909faf => generated 611 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:07.176089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c14c-763f-46f1-8f30-fd284fe3401f None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.177283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c14c-763f-46f1-8f30-fd284fe3401f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.177566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8c14c-763f-46f1-8f30-fd284fe3401f None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.179261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 209/428] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:07.185479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.185479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.185479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.220804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:07.222041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:07.255699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.257597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.356598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 210/429] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:07.383708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.420175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.424841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 220/430] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:07.431763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.493946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.521213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e82d1-b820-46df-a384-c38b75e08d10 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.552021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d8f78-3f45-438d-9827-e79ab98ea6e2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.566628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.567847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.568211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.568394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.579855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:07.588066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:07.589046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 12:53:07.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:07.591182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 211/431] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:07.596945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.604482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.605722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.606194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.606677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] PATH_INFO: `/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.610187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] RBAC: Authorizing `identity:get_project(project_id=c2fd0dd0cc73458992236ec1ddeb3150)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.612000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.614283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 221/432] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/projects/c2fd0dd0cc73458992236ec1ddeb3150 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:07.622655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.699136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.708282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a61d4-c91e-4204-b98e-6d5c4be51d90 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.741592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7531da6-2231-4a51-bd34-089752523883 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.745259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.746152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.746648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.747135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.748080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] RBAC: Adding query filter params (name=member) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:07.748625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.750404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.759994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 212/433] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:07.765608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.766780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.784050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.785135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.785243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.785482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.786730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.788266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.790302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 222/434] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:07.796620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:07.873753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b552ccc-9c52-4d70-b3af-35bfdab58381 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.911872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4ffb-0e05-4dfc-bb7a-b0f353cda597 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:07.919914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.920573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.920927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.921200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.949393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:07.965869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:07.966198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.968216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.968216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.968216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:07.968216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:07.969580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0dfb72-2d5d-4817-b9a0-9da0e68f90e4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:07.976382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 223/435] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=c2fd0dd0cc73458992236ec1ddeb3150&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:07.977684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba69b5-1944-4c11-ae14-12748ffa8aa9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.977971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba69b5-1944-4c11-ae14-12748ffa8aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.978250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba69b5-1944-4c11-ae14-12748ffa8aa9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:07.979533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 224/436] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:07.985635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:07.985920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:07.986203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.007944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:08.009924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:08.012391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.014087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.077814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.114799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 225/437] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:08.138652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.141195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 213/438] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:08.146220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.218732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.260925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6946-76cc-4fd8-b522-b67eabaa0123 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.275573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39775dd7-712a-4690-91bd-2fce162172af None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.305914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.306767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.307219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.307501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.313395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.314706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.315115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.315633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.317053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.318730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.319432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:08.322147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 214/439] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:08.328735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:08.329127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:08.329962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:08.330811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 226/440] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:08.337904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.391264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.459696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67880300-2277-4b4b-a489-1ca6f307bf64 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.465094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbfd03-55dd-4824-a1a0-4f20ed6ee8e7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.493247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.500888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.501563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.501839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.502107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.502883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:08.503187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.504563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.505808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.506699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.506977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.507248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.508429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.509869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aaca3e-8791-46c5-9141-7dbf550cee96 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.511541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 215/441] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:08.512742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc462a3-d3ec-49cb-8dbb-86d5458ce3b4 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.512997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc462a3-d3ec-49cb-8dbb-86d5458ce3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.517725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc462a3-d3ec-49cb-8dbb-86d5458ce3b4 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.517868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 227/442] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:08.517948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 216/443] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:08.518016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.518652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc462a3-d3ec-49cb-8dbb-86d5458ce3b4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.613753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ce17b-da95-46a7-ab97-3d939fc9460c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.616124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.616809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc462a3-d3ec-49cb-8dbb-86d5458ce3b4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.647357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.647983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.648267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.648531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.649381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=87f5d4315954429eb4783c57c3909faf, scope.project.id=ee8de25536f842af955b5f979fd05d27) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:08.649688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.650139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.650812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.651094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.651184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1667e4-86ed-4eca-9183-24bb3b3c59e5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.651367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.656932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 228/444] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=87f5d4315954429eb4783c57c3909faf => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:08.658160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.658523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.658801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.681955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:08.683267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:08.720720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.732317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:08.733906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:08.763880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 229/445] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:08.787056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.852438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 217/446] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:53:08.853355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.896730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.902026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3614e0-dadb-4a76-88b8-4fdef14f6a85 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:08.945917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:08.947027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:08.947306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:08.947574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:08.959701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:08.961473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:08.967902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:08.968807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:08.969570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 230/447] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:08.974998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:08.975418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6c1896-06a6-46da-8e71-73ef8e99d87c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.015674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.016526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.016800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.017069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] PATH_INFO: `/v3/projects/ee8de25536f842af955b5f979fd05d27` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.018245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] RBAC: Authorizing `identity:get_project(project_id=ee8de25536f842af955b5f979fd05d27)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.019713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.021407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 218/448] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/projects/ee8de25536f842af955b5f979fd05d27 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:09.027079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.077583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2aceb7-5d43-4b13-b313-e729797dfd6c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.113753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.114466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.114774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.115049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.115803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:09.116110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.117493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.124542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9ba6a-08d8-4699-a93a-5800df02b215 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.126181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 231/449] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:09.131725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.153598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:09.159817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.160664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.160942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.161209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] PATH_INFO: `/v3/groups/aca615027828472a8ad74f5a7b81cd0b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.162418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] RBAC: Authorizing `identity:get_group(group_id=aca615027828472a8ad74f5a7b81cd0b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.163850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.165504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 219/450] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/groups/aca615027828472a8ad74f5a7b81cd0b => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:09.171562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.198813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:09.232664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d250255-b4d3-43b2-b969-8f3e10e7d587 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.265404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.266055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.266350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.266659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.278283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d91da8-b872-4e1b-b0c0-dc0d5f575147 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.311958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.312644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.312920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.313177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.314000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] RBAC: Adding query filter params (group.id=aca615027828472a8ad74f5a7b81cd0b, role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=ee8de25536f842af955b5f979fd05d27) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:09.314311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.315787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.321471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 220/451] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/role_assignments?group.id=aca615027828472a8ad74f5a7b81cd0b&scope.project.id=ee8de25536f842af955b5f979fd05d27&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:09.333514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.335042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.358689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:09.445055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 232/452] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:09.451010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.545253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:09.556173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33196b-dbf3-4a4b-b056-3ddd647da124 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.590465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.591331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.591607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.591874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] PATH_INFO: `/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.593111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] RBAC: Authorizing `identity:get_project(project_id=288f97ff4fc34a4e979d3c926f4a9c7f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.594617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.596331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 221/453] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/projects/288f97ff4fc34a4e979d3c926f4a9c7f => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:09.601986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.704868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692474f-961c-40d8-b8ed-44862405ae63 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.725117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:09.741002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.742577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.742972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.743255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] PATH_INFO: `/v3/groups/fb1654cc07434c188fca5302a8b3dec6` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.744501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] RBAC: Authorizing `identity:get_group(group_id=fb1654cc07434c188fca5302a8b3dec6)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.745950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061eccc-fa9e-466a-83cd-70efe6715c5f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.747606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 233/454] 10.222.0.49 () {56 vars in 1180 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/groups/fb1654cc07434c188fca5302a8b3dec6 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:09.753953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:09.866212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48757e44-2e51-449e-ac24-fdae72bf00ee None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:09.904795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:09.905492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:09.905774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:09.906049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:09.906908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] RBAC: Adding query filter params (group.id=fb1654cc07434c188fca5302a8b3dec6, role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=288f97ff4fc34a4e979d3c926f4a9c7f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:09.907222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:09.908657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff3ee-236f-4a87-8d4f-97d0ecbbb327 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:09.914694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 222/455] 10.222.0.49 () {56 vars in 1376 bytes} [Thu Jun 30 12:53:09 2022] GET /identity/v3/role_assignments?group.id=fb1654cc07434c188fca5302a8b3dec6&scope.project.id=288f97ff4fc34a4e979d3c926f4a9c7f&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:12.336065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0a09-5b84-4194-bde5-d4fe1a44a1fa None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.336589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0a09-5b84-4194-bde5-d4fe1a44a1fa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.336589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab0a09-5b84-4194-bde5-d4fe1a44a1fa None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.338657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 234/456] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:12.343990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69155-a473-4faa-978f-19169f0e18e8 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.344751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69155-a473-4faa-978f-19169f0e18e8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.345166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f69155-a473-4faa-978f-19169f0e18e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.366698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-46f69155-a473-4faa-978f-19169f0e18e8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:12.368143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-46f69155-a473-4faa-978f-19169f0e18e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:12.392447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:12.460724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 223/457] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:12.508944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab0a09-5b84-4194-bde5-d4fe1a44a1fa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:12.612806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f6dd-0440-4d8b-af49-52a471680ad7 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.613453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f6dd-0440-4d8b-af49-52a471680ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.613945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85f6dd-0440-4d8b-af49-52a471680ad7 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.615881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 224/458] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:12.621175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a125-55bf-4704-a601-b99071d7f579 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.621704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a125-55bf-4704-a601-b99071d7f579 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.622223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a125-55bf-4704-a601-b99071d7f579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.638798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab0a09-5b84-4194-bde5-d4fe1a44a1fa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:12.653430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f5e9a125-55bf-4704-a601-b99071d7f579 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:12.654871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f5e9a125-55bf-4704-a601-b99071d7f579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:12.675103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:12.675898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.676177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.676450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.687770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:12.689248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a87d9f1a-1146-4c8c-93da-262f80a6647b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:12.708902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:12.709805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 235/459] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde853-b1c1-4254-a556-655611c14736 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde853-b1c1-4254-a556-655611c14736 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dde853-b1c1-4254-a556-655611c14736 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.715996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 236/460] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:12.716496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde853-b1c1-4254-a556-655611c14736 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:12.737324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 225/461] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:12.738980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.739540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.740036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.757284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:12.758826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:12.762736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:12.831319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 226/462] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:12.832578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:12.833844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dde853-b1c1-4254-a556-655611c14736 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:12.848292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:12.875960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:12.876556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.876837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.877112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:12.877854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:12.878170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:12.879872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:12.893159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 237/463] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:12.894363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:12.951604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2744ef7f-55dc-4578-b8c3-41d7bd098095 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:12.988728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:12.989581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:12.989861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:12.990177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.001029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.002545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.020808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:13.021999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:13.022880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:13.023699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:13.023699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:13.023699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.023699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:13.023699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 227/464] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:13.028811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.028944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b393-e26f-4520-881b-f2bb35ef4e7b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.063497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.064423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.064766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.065107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.076601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.078542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:13.098601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:13.100318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:13.102278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:13.102278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:13.102278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.102278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:13.102278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 238/465] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:13.102926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.105919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.149631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53121b45-7624-483c-ae69-fcff8de4c29f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.189007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.189680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.189961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.190235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.190988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.192461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.218627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 228/466] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:13 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:13.220135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.267797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf3cac-414c-4f13-8fac-e2add13b33de None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.341511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c920d01-c4e7-4e9e-aad8-dea5781f9a82 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.350981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.351653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.351940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.352313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.353063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:13.353373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.354923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.367482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.382647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.383675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.384100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.384411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 239/467] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:13.384701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.385672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:13.386121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.387684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f19af3-21d0-491d-bfe3-cd4c45200185 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.391141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.403684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 229/468] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:13.405503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93900e-e685-4058-8057-4c79fec19ff2 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.406012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93900e-e685-4058-8057-4c79fec19ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.406482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93900e-e685-4058-8057-4c79fec19ff2 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.408455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 230/469] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:13.409412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93900e-e685-4058-8057-4c79fec19ff2 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.410205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.507285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315cb2-1719-440c-8281-6737c32f688c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.540168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93900e-e685-4058-8057-4c79fec19ff2 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.555214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.555893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.556286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.556460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.557178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.558713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f84258-0663-4a4f-9d64-d7883ef40368 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.582253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.583141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.583534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.583875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.584707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.586187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 240/470] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:13 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:53:13.586309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.588283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.588754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.589635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.611351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:13.612990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:13.614175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 231/471] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:13 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:13.694836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.700409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 241/472] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:13.729160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.772254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:13.849102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5dcac-87b3-42e6-942f-dc7abbf2dbb2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:13.888201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:13.889010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.889371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.889665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.902164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:13.903705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a3bd1346-626a-4a66-a1fe-2ce9eaa1ba8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:13.922449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:13.923397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:13.924217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:13.924217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:13.924217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:13.924217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:13.926508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 232/473] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:13.932163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:13.972186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0019-d967-4358-a669-65042364dd74 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.972487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0019-d967-4358-a669-65042364dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.972667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b0019-d967-4358-a669-65042364dd74 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:13.974188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 233/474] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:13.980494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:13.980858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:13.981105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.001058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:14.002524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:14.048984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b556e11-a6f3-48e7-99e0-742cff4520c1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:14.098773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 234/475] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:14.100614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:14.101293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:14.101573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:14.101847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.102628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:14.102942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:14.104415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:14.117092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 242/476] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:14.122618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:14.134157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:14.154679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:14.247291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:14.294617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c46bb5-14c6-46f6-afec-b10e00debc10 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:14.332299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:14.333401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:14.333923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:14.334434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.335348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:14.336995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:14.357683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43999699-0453-478f-86a6-cb49ec597d72 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:14.374135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 235/477] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:14 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:53:14.394116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:14.394873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:14.395155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:14.395430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.396207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:14.397619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:14.407684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.sql.core [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:53:14.407684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 12:53:14.407684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [parameters: {'id': '9813f034e1054f87b77e0f5913852806', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 12:53:14.407684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80102) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result.read() Jun 30 12:53:14.432338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:53:14.433270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application yield resource Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise self.value Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application session.commit() Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:53:14.434246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:53:14.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:53:14.435999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application result.read() Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:53:14.436956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [parameters: {'id': '9813f034e1054f87b77e0f5913852806', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:14.437730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 12:53:14.440537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:14.444258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 243/478] 10.222.0.49 () {60 vars in 1131 bytes} [Thu Jun 30 12:53:14 2022] POST /identity/v3/roles => generated 144 bytes in 300 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:14.444414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:14.558213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:14.586745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b59a35-71bd-4d06-ab90-148eadcb9219 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:14.629824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:14.630682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:14.630951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:14.631220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.642219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5f04b17d-96a2-430d-b84f-530eeec8a690 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:14.650536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:14.651496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:14.652272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 236/479] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:14.659438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:14.768829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13402ee-c27e-443a-ba3b-569efe5e444a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:14.794362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:14.807277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:14.808015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:14.808705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:14.809138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:14.809904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:14.810216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:14.811683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fabe2-cadb-4537-9439-4210b43ef8f6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:14.820690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 244/480] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:15.633714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7315a3-e004-4fb7-ad53-74ba72e544b4 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:15.634240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7315a3-e004-4fb7-ad53-74ba72e544b4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:15.634240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7315a3-e004-4fb7-ad53-74ba72e544b4 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:15.635932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 237/481] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:15.642028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:15.642966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:15.643452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:15.671381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:15.672911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:15.759998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 245/482] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:15.795840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7315a3-e004-4fb7-ad53-74ba72e544b4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:15.939255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7315a3-e004-4fb7-ad53-74ba72e544b4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:15.985755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:15.986606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:15.986888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:15.987165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:15.992295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.006898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-854d779a-9aef-4054-9832-cebfa9f5052b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:16.014922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:16.015790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.016547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:16.017567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 238/483] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:16.024237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.121620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df88c9-4797-4306-9549-ce1aa1587888 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.122303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df88c9-4797-4306-9549-ce1aa1587888 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.122714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df88c9-4797-4306-9549-ce1aa1587888 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.124823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 239/484] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:16.129501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.129787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.130061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.147280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:16.150158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:16.189652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a65601-232e-4a82-ba74-bfbc531e7055 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.205671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.245477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.246154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.246662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.246938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.247692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:16.248000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:16.251002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:16.274364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 240/485] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:16.279800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 246/486] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:16.285312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.314261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.405751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cdf6b-344c-4468-ab5a-408d4e9d979d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.433048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.449657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.450949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.451393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.451855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.452399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663f69-65af-4b39-8900-779c3bb764d1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.466645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:16.469300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:16.490818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.491759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.492138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.492506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-1cdd7b42-b6c5-4200-b214-115044575336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:16.494580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:16.495399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:16.496227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:16.496227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:16.496227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:16.496227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:16.498476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 241/487] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:16.504506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2d57-022b-4a2b-86dd-85b4be252959 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.504506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2d57-022b-4a2b-86dd-85b4be252959 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.504506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d2d57-022b-4a2b-86dd-85b4be252959 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.505729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 242/488] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:16.507128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d2d57-022b-4a2b-86dd-85b4be252959 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.517572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b225118b-ff23-453e-b22d-030ab275384b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:16.526237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:16.527355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:16.528304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 247/489] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:16.533737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.533737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.533737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.549768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:16.551496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:16.585351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.625720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d2d57-022b-4a2b-86dd-85b4be252959 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.638598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 248/490] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:16.640053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.667047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.667754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.668029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.668300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.669024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:16.669330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:16.670783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:16.686432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 243/491] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:16.687279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.722928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.759654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0935bb3-a452-4059-a03f-bf283639896c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.788493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9633c-205b-4af7-bbb8-d9cb7a0912ca None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.797617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.798671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.799277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.799770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.800717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:16.801216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:16.803107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:16.817373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 249/492] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:16.819009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.822823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.823821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.823928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.824170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.834769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:16.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:16.844084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:16.844862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:16.849321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 244/493] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:16.850484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:16.855287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.936534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae367-fbe2-4271-a2c4-5d3513fcd152 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.945915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:16.951963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0b446-8c31-490f-952e-3b847fccd3a8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:16.977521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.978768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.979361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.979824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.988017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:16.988760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:16.989039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:16.989309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:16.989709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:16.991508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.000822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.002420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:17.009237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:17.010143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:17.011034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:17.013423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 250/494] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:17.014827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:17.020860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:17.021892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:17.022963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:17.022963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:17.022963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:17.022963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:17.028690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 245/495] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:17.028690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.118384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.131523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a10c-2b49-4bad-b1b2-14fa04c59f9f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.136289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8d45-4d9f-45e9-b91c-8af89afdd40e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.168198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.168900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.169185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.169464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.170230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:17.170623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.172034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.172206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.172918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.173290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.173476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.174361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:17.174681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.176098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc5f99-f979-41fb-97d6-14faafd4ba7f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.181059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 251/496] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:17.181702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.218469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 246/497] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:17.220197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2041-dcb4-449d-8acd-723618ad95ff None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.220482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2041-dcb4-449d-8acd-723618ad95ff None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.220759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c2041-dcb4-449d-8acd-723618ad95ff None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.222148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 247/498] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:17.226434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2041-dcb4-449d-8acd-723618ad95ff None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.233179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.299744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6290f-4e92-4f15-97ff-8758561ef9b3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.335073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.336006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.336436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.336714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.337783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:17.338080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.339513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.339949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c2041-dcb4-449d-8acd-723618ad95ff None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.352059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 252/499] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:17.352710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.365989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.384169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.384962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.385292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.385553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.395965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.397438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-f40720ee-cd3d-49ee-bea8-924e0fdb772d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:17.416245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:17.417192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:17.418010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:17.418010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:17.418010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:17.418010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:17.418010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 248/500] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:17.419092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.419092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.419275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.435968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:17.437523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:17.465571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53709541-c3eb-482c-9648-324c61bec906 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.489999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.498435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.499182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.499469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.499744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.503965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 249/501] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:17.504789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.568928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.570436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a1c2a-0a7a-446d-8dbf-79245a281dee None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.581892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.602820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287434f-c859-4c85-97f0-2b6da2e80153 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.636308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.637101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.637380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.637653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.646439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.647912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:17.663807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:17.664697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 250/502] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:17.671535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.674081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 253/503] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:17.675489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4565-843b-4e34-b81b-97c0dd4578a8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.675762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4565-843b-4e34-b81b-97c0dd4578a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.676032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee4565-843b-4e34-b81b-97c0dd4578a8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.677571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 254/504] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:17.678260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee4565-843b-4e34-b81b-97c0dd4578a8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.721896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.772245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef23d0e-b2ae-4d74-afec-e37dd697c44e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.781026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee4565-843b-4e34-b81b-97c0dd4578a8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.791723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.806086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.806848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.807137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.807411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.808128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:17.808434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.809825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.815461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.816259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.816546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.816817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.821369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 251/505] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:17.825692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.832108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:17.840276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:17.841211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:17.841985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 255/506] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:17.842820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:17.931393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f336b6-b211-4dc7-a5a0-57ca411a4d8b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.940678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:17.952961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab238b3d-2d57-4359-8fc1-e7b8bdb03a3c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:17.968177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.968849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.969132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.969407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.970253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:17.970601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:17.972055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0a896-f1d5-4f89-9fca-8bb01dc695dc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:17.986563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:17.987541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.987822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.988096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:17.990092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 252/507] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:17.991350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:17.991640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:17.991917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.003770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.005296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.007364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:18.008568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:18.022967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.023812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.024667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:18.024667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:18.024667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.024667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:18.024667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 256/508] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.024924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.049325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.083809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 253/509] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:18.084535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.137732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb74e-df14-4a02-acdb-aa644e9f6539 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.171227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.172018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.172294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.172562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.178878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.181055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.182607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.184913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f504ccf-f9f2-4ffa-b20d-889b495e0c2a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:18.197066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.197950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:18.198891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 257/510] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.199431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.224510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.224978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.225261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.225536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.226289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:18.226630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.228008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.236115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 254/511] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:18.236947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.291935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.309742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4049f-c833-4917-8889-89dbac0f74e6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.339205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6600d6-3ae3-4772-b3eb-ee9070311ee3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.343301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.343953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.344238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.344515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.375537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.376275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.376624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.376950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.377690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:18.377993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.379398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.391054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 255/512] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:18.391844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.410622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.415136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.416628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.489638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca081f6-48a8-4ebf-8f0d-661a1f9b9b5d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.493593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.513396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 258/513] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:18.514304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.529999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.530904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.531183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.531452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.541792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.543319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:18.560951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.561953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 256/514] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.567984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.616537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2a456-8540-44b7-8bda-5e9845b1b84f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.634081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.649761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.650592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.650875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.651147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.651974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:18.652282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.653720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.664025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5c81-4e9c-4e5f-ac5a-87a0f7a84621 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.688800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 259/515] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:18.689604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.701365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.702239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.702549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.702824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.713326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.714871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.730914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:18.733502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.734383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.735371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:18.735371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:18.735371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.735371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:18.739690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 257/516] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.740398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.789865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88935c2f-ee18-4658-a97f-4bd89ce86b2e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.823302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.824027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.824312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.824588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.831514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.833046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.836330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7936bdd-77ac-45ed-a9d2-6b97c7929f64 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:18.848239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.849100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:18.849950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 260/517] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.850419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.862425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.874735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.875556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.875834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.876098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.886341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.887921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:18.905822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:18.906679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:18.907498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:18.907498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:18.907498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:18.907498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:18.910477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 258/518] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:18.911548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:18.950889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:18.955136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb11e7-8804-4880-9393-73aa1f8df4ef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:18.988662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:18.989317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:18.989599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:18.989866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:18.990622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:18.990929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:18.992326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.004008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 261/519] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.004806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.015399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d6b1-9ff8-481d-8579-704c2d363d4a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.048283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.048925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.049232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.049505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.091694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.099562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85306005-a971-49d1-8e5e-d121a6922f32 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.116467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.117932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.132307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.132962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.133245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.133521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.134231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:19.134569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.135925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.147114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 262/520] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.147904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.169364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.220690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 259/521] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:19.221534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.247169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1569ab-7d31-4052-8006-b803e272c5ba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.286872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.287526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.287788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.288080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.288855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:19.289177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.290647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.297380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.309461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 263/522] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.310262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.326147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee53def-7463-4a9e-a852-a546fbec30d1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.362696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.363359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.363641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.363914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.364636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:19.364944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.366346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.377683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 260/523] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.378497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.389470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.409527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673b56f-bc9c-4835-b102-e12475d565d9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.442622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.443293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.443577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.443850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.444559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.446046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.468304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 264/524] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:19 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:19.469113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.490699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541892c-36e3-474a-ae25-d1e2ea88e590 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.530765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.531571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.531856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.532124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.535056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.543375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.544819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:19.561896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:19.563057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:19.563931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 261/525] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:19.564467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.585842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375859-832b-4a6e-bc42-356301b27466 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.625328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.626182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.626469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.626896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.629721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.639590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.641420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:19.662433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:19.663568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 265/526] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:19.664430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.689918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64558721-33f1-4eb7-b6cc-69fd5e9833a4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.726608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.727258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.727633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.727905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.728720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:19.729026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.730483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.736107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 262/527] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.736943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.789648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea3de-f444-4920-95c8-3d205143cca5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.815094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.829362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.830172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.830533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.831265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.839209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.840875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:19.857861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:19.859504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:19.860635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:19.861685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 266/528] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:19.862787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.880039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89337c2-6dbd-487c-83a8-ee6db003b0fe None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:19.920909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:19.930789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:19.931656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:19.932025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:19.932421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:19.933711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:19.933958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:19.935535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:19.956580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 263/529] 10.222.0.49 () {56 vars in 1139 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:19.957862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:19.977360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab276b-9433-4bdf-a0da-6f38c800f7c1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.015169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.016113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.016537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.016950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.017932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:20.018484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.020025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.033091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 267/530] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:20.034139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.073704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8479cd-8713-4151-a78f-a25388b6bf72 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.093014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.119471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.120763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.121216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.121877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.122864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:20.123651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.125839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.145661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc88ec-0ebd-4470-8924-f583f6bc6c14 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.150078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 264/531] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:20.151187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.163307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.201525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.202493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.202918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.203361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.283274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.285023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.297318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66920-52bc-473c-9e30-334599423fa8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.344307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.345215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.345567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.345908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.353475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.355055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:20.370040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:20.371134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:20.372172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:20.373535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 265/532] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:20.374496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.400413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 268/533] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:20.401536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.412123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.413159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.481903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613f926-37e0-4772-823a-9379963ff3ba None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.504423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf06ecb-ccac-4253-8ba0-2ceaf3696443 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.518362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.519335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.519866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.520337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.521360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=607bae995849489db891582db395e28c) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:20.521872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.523559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.538647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 266/534] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/role_assignments?user.id=607bae995849489db891582db395e28c&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:20.539984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.540351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.546350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.558309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 269/535] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:20.564477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.657223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.661466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a0c36-d709-4c20-8a3b-1ab8d4d532b7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.702826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.703893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.704351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.704771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.717659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.719553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:20.738211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:20.739564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:20.740383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:20.740383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:20.740383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:20.740383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:20.740383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 267/536] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:20.749123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:20.769494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:20.830811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abb7bc-d7b3-4162-96d0-fb36024dcf76 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:20.885139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:20.885822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:20.886567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:20.886855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:20.887729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=aaf234ea529642af8039f8e3b0bf601c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:20.888039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:20.889774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:20.902556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 270/537] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/role_assignments?user.id=aaf234ea529642af8039f8e3b0bf601c&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:20.950760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf38156-ef6f-4bd1-ae99-c6b694654de9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:21.006922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:21.007603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.007886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.008158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.008897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:21.009204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:21.010618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:21.024797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 268/538] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:21.030769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:21.163403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4c260-ae5d-421b-8020-275d8dc1029b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:21.199736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:21.210788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:21.211531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.211812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.212436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.223772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:21.225252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:21.244628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:21.245473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:21.246314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:21.250431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 271/539] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:21.258364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:21.282771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:21.421370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e0eb2c-4229-4ca3-bd83-848f185c8efd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:21.471115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:21.471704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.471988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.472261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.473052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:21.473360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:21.474828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a9a91-670c-4415-8107-c40187dff997 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:21.487662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 269/540] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:21.493600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:21.595201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce734cb-a65d-4fc7-9399-11ccb01b04d7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:21.641935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:21.642622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.642908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.643183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.644021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=6e59c5b4dfe04f30a195ab3e83f49c2a) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:21.644332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:21.645793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7dcf7-d0bd-4b0a-8599-b17d1775c987 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:21.654447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 272/541] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:21 2022] GET /identity/v3/role_assignments?user.id=6e59c5b4dfe04f30a195ab3e83f49c2a&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:21.757066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64da0-f572-416b-bdd7-456645451508 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.757563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64da0-f572-416b-bdd7-456645451508 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.757563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b64da0-f572-416b-bdd7-456645451508 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.759080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 270/542] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:21.769487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:21.769836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:21.770053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:21.796889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:21.798171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:21.871411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:21.891420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 273/543] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:21.915041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b64da0-f572-416b-bdd7-456645451508 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.003902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:22.045986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b64da0-f572-416b-bdd7-456645451508 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.092478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.093643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.094136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.094618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.110090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-f5cc4679-0451-4b0c-9c8f-57e7088c7b5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:22.118126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:22.118999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:22.119765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:22.122455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 271/544] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:22.128831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.183606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3926a-46ec-43cc-8eed-aee866a59fa6 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.184227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3926a-46ec-43cc-8eed-aee866a59fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.184713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3926a-46ec-43cc-8eed-aee866a59fa6 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.186733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 272/545] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:22.192685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.192976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.193250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.222674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:22.223888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:22.235720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dc08a-9a32-4c72-b895-7fc266a2e45c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.281286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.282344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.282854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.283364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.284426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:22.285069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:22.286756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:22.310682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 274/546] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:22.312295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.319708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 273/547] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:22.348488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.396789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:22.477790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:22.487144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71869f8-d563-48f7-b0b7-ffd6d4131a2c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.548766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.549594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.549878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.550152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.553737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182e749-019b-4f4e-818c-a87d48e54978 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.574564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:22.576716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:22.604460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:22.605345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:22.606151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:22.606151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:22.606151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:22.606151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:22.606812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 275/548] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:22.612928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.619181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.619994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.620272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.620542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.642929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:22.650736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:22.651682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:22.652939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 274/549] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:22.659140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.735894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:22.739965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b787f-35c5-4686-9237-bc6aecc8dcec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.783240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.784353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.788801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.789302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.790225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:22.790745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:22.792336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:22.817681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 276/550] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:22.831728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:22.913482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25982762-eb45-4e67-ac59-1184af18c6fd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.937494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:22.949911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:22.951155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.951658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.952364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.953358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:22.953877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:22.955486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc95e0-19de-4286-bf06-bf5067fa7735 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:22.965857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 275/551] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:22.969460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82c75-533f-49a6-b922-7963443f84cf None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:22.971203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c1f7-831b-4348-a659-32774392cddf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:22.972091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82c75-533f-49a6-b922-7963443f84cf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:22.972775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c82c75-533f-49a6-b922-7963443f84cf None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:22.975145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 276/552] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:22.976100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c82c75-533f-49a6-b922-7963443f84cf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.011058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.011873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.012214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.012459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.020982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.022726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fcaf503f-6343-417a-9c50-891aa35f6413 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:23.039206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:23.040170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:23.041169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:23.045442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 277/553] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:23.047452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.047746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.048024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.066225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:23.067455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:23.083573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.091313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c82c75-533f-49a6-b922-7963443f84cf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.127744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.128921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.129391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.129851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.140397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.142125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.143507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 278/554] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:23.151143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:23.161362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:23.162245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:23.163458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:23.163458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:23.163458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:23.163458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:23.165491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 277/555] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:23.171684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.211255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.276892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbba1fb-1af8-4307-986c-d893b9a0ce9d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.278700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463d88d-39e6-4eb8-ae2d-bfd39235364a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.316459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.317254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.317601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.318023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.318160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.318562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.318729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.319003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.319830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:23.320009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:23.320298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.320591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.321845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.322360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.338660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 278/556] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:23.339595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.347120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 279/557] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:23.348258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.360225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.440831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.447627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173adb44-a39e-461c-af0c-d949a0649747 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.457180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a1167-b526-42cc-95ba-1ed6047c188b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.484409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.485186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.485553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.485952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.496857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.497347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:23.498015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.498628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.499106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:23.506031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:23.507649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:23.508650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:23.510155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 279/558] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:23.511036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.515547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.517072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1fc1cfa1-af30-466c-a860-3065a656418a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:23.532732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:23.533716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:23.535173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 280/559] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:23.535749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528e59-eee1-44df-85f8-c48df8479c41 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.535749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528e59-eee1-44df-85f8-c48df8479c41 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.535874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c528e59-eee1-44df-85f8-c48df8479c41 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.541104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 281/560] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:23.541104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528e59-eee1-44df-85f8-c48df8479c41 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.611159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.627707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5aab0-6872-4063-b3a5-298506c4b41e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.649625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c528e59-eee1-44df-85f8-c48df8479c41 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.664362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.665013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.665524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.665664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.689941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.691784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.692807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.692923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.693217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.695021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:23.695351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.696726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.710460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 282/561] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:23.711509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.746194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.747834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eb9f2-7eae-4541-9bbe-4f670d967966 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.814677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6b32d-1d2e-492f-a5f2-4fda644ba1d1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:23.844579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.852844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:23.857706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 280/562] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:23.857907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.857907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.857907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.858173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:23.858287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:23.858450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:23.858680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:23.858814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:23.858944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:23.875101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 283/563] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:23.876339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:23.876718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.877550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:23.931264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:23.948602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 281/564] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:23.951201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:23.986714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad67e2-c23c-4f1d-b17f-64e0e9e3852b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.020152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.020968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.021242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.021487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.031542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.033000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:24.050644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.051553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 284/565] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.052613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.057652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916a67-7ba7-41c6-ba33-99539c892584 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.092496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.093380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.093655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.093925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.094848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.104147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.105580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-94a699db-e423-425d-b61c-d9c643806613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:24.123408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.124559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.125500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:24.125500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:24.125500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.125500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:24.125500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 282/566] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.126454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4099-691c-4c09-aef7-e046d5a7fa1c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.126738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4099-691c-4c09-aef7-e046d5a7fa1c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.127011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe4099-691c-4c09-aef7-e046d5a7fa1c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.130356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 283/567] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:24.130356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe4099-691c-4c09-aef7-e046d5a7fa1c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.153952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f1586c-ac39-4865-ae1e-24b6ba27fa55 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.171381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.187073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.187874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.188146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.188414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.199003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.200475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:24.218346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.219223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.220101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:24.220101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:24.220101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:24.220101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:24.220101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 285/568] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.220365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.236881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe4099-691c-4c09-aef7-e046d5a7fa1c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.275093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.275897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.276176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.276448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.287136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:24.294891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.295840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:24.296739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 284/569] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.297406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.315492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.324306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbcf54-d222-429f-a58f-67caaeac15d6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.358199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.358887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.359163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.359431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.360149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:24.360455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.361836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5275966c-2e28-451e-865d-ce2e6a6ba568 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.373904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 286/570] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:24.375148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.375400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.375723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.391596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:24.392787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:24.399140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.403026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b499448-fc01-4bf8-8769-bcbb30758632 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.437830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.438535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.438811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.439081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.439807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:24.440110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.441479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.453051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 285/571] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:24.453826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.466625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 287/572] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:24.467535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.555555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e122-b0f0-4cc2-8c94-1298c5a3bbdf None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.556978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.577600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd162f-82c8-4484-9b3f-20cfbb5f1893 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.588842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.589847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.590121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.590415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.591134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:24.591434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.592929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.604592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 286/573] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:24.605382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.614735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.615409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.615699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.615974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.616725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:24.617034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.618490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.624165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.627942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 288/574] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:24.628674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.706023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90089276-5c9b-4e46-916a-35dfa78897bd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.729193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533df1c-f1e7-45c0-baf4-9f29feac7700 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.740261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.741049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.741331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.741604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.748944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.750384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.762682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.763500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.763773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.764039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:24.764136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.765004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 287/575] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.766217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.772950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.774449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:24.789791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.790861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 289/576] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.791896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.798802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.852438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:24.873992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8628a6-cef2-4f56-be82-7b55a24168c4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.898172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3170430-dc61-4e67-8c6f-a3991fa5ae45 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:24.907054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.907843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.908119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.908388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.918620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:24.926288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.927423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 288/577] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.928428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:24.935287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:24.936161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:24.936442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:24.936710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:24.943666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:24.945187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:24.959886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:24.960783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:24.961634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:24.966223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 290/578] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:24.966998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.028411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.031057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd5bc7f-d5d1-46a2-bdc8-081e498b55da None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.066709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.067250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936cd0b-bf7d-4652-92a9-0a57d9a36409 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.067502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.067781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.068052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.074759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.080094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.081565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:25.098881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:25.099771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:25.100707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:25.100707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:25.100707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.100707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:25.100707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 289/579] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:25.100976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.101073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.101274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.101460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.101752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.102576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.102885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.104321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.122440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 291/580] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.123367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.208175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9fc0f-be28-4842-b3c4-39921f86bdda None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.222620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0047c-debb-4d23-b1b0-cdef24845815 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.242967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73200db3-d474-4a56-8862-310bec1930be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.243620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200db3-d474-4a56-8862-310bec1930be None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.243900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200db3-d474-4a56-8862-310bec1930be None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.244169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73200db3-d474-4a56-8862-310bec1930be None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.244955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73200db3-d474-4a56-8862-310bec1930be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.245091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.245261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73200db3-d474-4a56-8862-310bec1930be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.246705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73200db3-d474-4a56-8862-310bec1930be None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.258906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 290/581] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.260169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73200db3-d474-4a56-8862-310bec1930be None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.265914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.266619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.266896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.267164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.267920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.268223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.269615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.278569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 292/582] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.279617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.331346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.386456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73200db3-d474-4a56-8862-310bec1930be None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.395368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98582b9d-dcda-4894-a3d7-dd1e6459245a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.420441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.421190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.421473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.421745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.422576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.422913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.424346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.431326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.432455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.432739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.433016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.433742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.434050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.435953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.438400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 291/583] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.439367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.450478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 293/584] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.451375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.496455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.553980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04afd-37f2-4d6d-9e86-4b10db6e0054 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.554423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549ced7-94d5-4934-80bb-3c2b7d4d3e41 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.568708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.593557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.594218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.594611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.594880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.595084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.595710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2b2d6693e40d40f988254bbc7fcca45f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:25.595886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.596015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.596163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.596430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.597460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.602868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 294/585] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/role_assignments?user.id=2b2d6693e40d40f988254bbc7fcca45f&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:25.603655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.613609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.615237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:25.635436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:25.636586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:25.637607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:25.637607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:25.637607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.637607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:25.637607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 292/586] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:25.643039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.726302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010063-d623-4f4d-a25d-2374f3238d1f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.737256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.765677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49735be2-2d05-43f6-b441-61f348df0f20 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.768692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.769660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.769660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.769660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.780463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.782066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:25.799358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:25.800259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:25.801112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:25.801112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:25.801112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:25.801112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:25.801112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 295/587] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:25.802800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.803689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.803963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.804230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.805848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.823358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=2a0b0da6c54341ff8fb66f48a0a32b70, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.825472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:25.840761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 293/588] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:25 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:25.855326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:25.880123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:25.936939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4dab-5dca-4231-b270-36952ad9e50e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.977692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:25.978806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:25.979185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:25.979769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:25.983157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226c239-1923-4826-a3e0-02f4ccd38d23 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:25.988929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:25.990594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:26.007923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:26.009099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:26.010013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:26.010658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 296/589] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:26.011255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.031337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.031927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.032205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.032475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.033261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:26.033557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.035085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.036375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.048456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 294/590] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:26.049390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.132749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccaa767-8a71-46e9-9263-b2c53917ccd3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.137897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.169214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555bd63-4c99-4663-ae8d-700e956a342e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.180351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.181073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.181360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.181774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.182541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:26.182851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.184267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.202410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 297/591] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:26.203310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.235346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.236023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.236298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.236568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.237398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:26.237705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.239169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.262860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 295/592] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:26.268370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.318756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.346636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11c35e-35f5-41bf-9b3a-a5dc336b2d1b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.374064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7598fe-ceaf-4364-ad72-e39f882f19f3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.391685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.393303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.393642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.403638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.403922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.407162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.410350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.411559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.412203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.412564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.420335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.421876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:26.426466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:26.427495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:26.428588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:26.430890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 298/593] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:26.432041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:26.439838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:26.440828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:26.441734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:26.445949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 296/594] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:26.446912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.549871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a880d86-0360-4153-a243-39508bed67c3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.564162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf9d72-516f-4ea1-becf-f2a1f25a66bd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.589323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.590037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.590402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.590738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.591584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:26.591951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.594370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.609168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.609650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 297/595] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:26.610565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.610673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.611270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.611775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.623234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.627559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.629747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:26.650278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:26.651219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:26.652079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:26.652079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:26.652079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:26.652079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:26.656216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 299/596] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:26.657059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.671887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.727512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b4ff-ecce-44b7-98b3-a99e61c8011d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.761336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499fc8-54cd-4a15-b277-689ff259a222 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.765519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.766481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.766961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.767503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.768665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:26.769172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.771006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.790474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 298/597] 10.222.0.49 () {56 vars in 1139 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:26.791668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.800009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:26.800907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:26.801180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:26.801447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/607bae995849489db891582db395e28c/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:26.817142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=607bae995849489db891582db395e28c, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:26.818624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:26.836234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 300/598] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:26 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/607bae995849489db891582db395e28c/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:26.842662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:26.934176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:26.978868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740ed83-100e-4766-a15d-985517431e40 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:26.993620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:27.054552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.055227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.055510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.055778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.056540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:27.056828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.058232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.059497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bc691-e7ec-414c-9b79-d64036985181 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.076880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 299/599] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:27.078203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.106374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.108147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.108929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.109406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.127466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.129744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:27.154644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:27.155734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:27.156602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:27.156602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:27.156602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:27.156602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:27.159068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 301/600] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:27.166611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.199600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294247-d594-4e3c-9b6a-d66782d8f82a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.235319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.236159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.236438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.236705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.243821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.245309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:27.262206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:27.263072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:27.263931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:27.273376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 300/601] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:27.274702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.278907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:27.285964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a764f1-2909-4945-9426-d2dff2b03a16 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.304260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:27.322801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.323843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.324323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.324822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.325805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:27.326342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.328283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.341277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 302/602] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:27.359631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.389402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a507073-4588-45bb-9b84-d38a4c923a92 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.426897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.427577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.427859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.433656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.433656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:27.433656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.433656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.449066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 301/603] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:27.450367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.545119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dee0b8-da1a-4dd4-9700-f6bdd52e2de2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.547482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:27.576108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9e09a-1c78-4fa9-a46d-c12321266f9d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.614486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.615213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.615495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.615767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.624960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.626480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:27.642964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:27.643905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:27.644772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:27.649192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 303/604] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:27.650362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.655140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.656054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.656327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.656594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/aaf234ea529642af8039f8e3b0bf601c/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.667399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:27.711746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=aaf234ea529642af8039f8e3b0bf601c, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.713559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.738242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 302/605] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:27 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/aaf234ea529642af8039f8e3b0bf601c/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:27.772133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dfa2c-4aba-488b-a7ed-f251eda2492e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:27.822097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:27.823051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.823506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.823935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.824872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:27.825325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:27.827317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd758f-9a6b-4db3-ba7d-f5c9b3d47df1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:27.841521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 304/606] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:27.848200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:27.964592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38efa-acc8-4f79-8f53-d40fbd8440eb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.965147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38efa-acc8-4f79-8f53-d40fbd8440eb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.965440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b38efa-acc8-4f79-8f53-d40fbd8440eb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:27.967345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 305/607] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:27.972572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8807a4-4e5a-43a2-9e72-33c767a2228c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:27.972860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8807a4-4e5a-43a2-9e72-33c767a2228c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:27.973138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8807a4-4e5a-43a2-9e72-33c767a2228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.010038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ab8807a4-4e5a-43a2-9e72-33c767a2228c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:28.013874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab8807a4-4e5a-43a2-9e72-33c767a2228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:28.027739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c37c-005f-40af-83e6-83cdd656ea7e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.079069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.081856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.082349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.082808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/6e59c5b4dfe04f30a195ab3e83f49c2a/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.094235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.103547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=6e59c5b4dfe04f30a195ab3e83f49c2a, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:28.105615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:28.114641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 306/608] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 12:53:28.116612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad7eeb-4083-47d5-8da0-eaa91abea936 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.117110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad7eeb-4083-47d5-8da0-eaa91abea936 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.117562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad7eeb-4083-47d5-8da0-eaa91abea936 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.119263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 307/609] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:28.125176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 303/610] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:27 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/6e59c5b4dfe04f30a195ab3e83f49c2a/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:28.126974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.127569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.128023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.134990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.147837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.154541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:28.156286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:28.233412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 308/611] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:28.298218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9ce5-404e-411a-9c55-1312c406d6cd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.301346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.336383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.337535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.337810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.338076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.349283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:28.358966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:28.359951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:28.360772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 304/612] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:28.366626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.420099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.430622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca495e21-b466-4699-9a74-b49efc3874b3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.470497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.471717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.472303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.472920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.475399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.479530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c16f5e-32de-40c7-a861-9ee4be0a5c2e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.485736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:28.493785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:28.494722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:28.495558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:28.498240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 309/613] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:28.507557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.516369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.517036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.517314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.517584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.522342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:28.522342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:28.522342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:28.529134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 305/614] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:28.538355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.636682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a8acc-4721-4f8e-aa01-6e7db039b3cd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.647484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.673278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.674448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.674903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.675343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.686680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:28.688361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:28.692587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada3952-0da7-4bb8-81d0-b9543a60576c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:28.706614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:28.707543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:28.708408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:28.708408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:28.708408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:28.708408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:28.710734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 306/615] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:28.716701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.779161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.780112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.780422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.780680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.781472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:28.781780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:28.783217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:28.805436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 310/616] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:28.811719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.848693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02a1ba-9fc0-4824-b356-41cedc9a3636 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:28.904107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:28.904814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:28.905094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:28.905366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:28.906037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:28.906145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:28.906404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:28.908181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:28.934934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 307/617] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:28.947286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:28.994816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.001263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877370ad-4352-4e64-94e3-e13ec9dcf6b5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.074549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.075459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.075580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.075803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.089277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db2786-bc6c-403a-9c0f-42468ff40dfb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.098662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.100132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:29.122149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:29.123665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:29.124756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:29.124756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:29.124756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:29.124756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:29.132073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 311/618] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:29.132073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.140796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.142140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.142831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.143271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.151741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.153468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:29.168853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:29.169878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:29.170853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:29.173416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 308/619] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:29.179156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.272015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.286217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744dcf7e-07b0-4070-9189-9b259700e954 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.303939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b803-8cfc-42dc-95a2-03aedaeba14f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.315388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.330135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.330761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.331041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.331338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.332256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:29.332569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.334016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c9245-932a-4507-abba-155458b1f04c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.349476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.351351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.352023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.352549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.353455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:29.354088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.355013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 309/620] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:29.355918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.356575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cbb2-7883-4ccf-a917-605a391b6394 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.356953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cbb2-7883-4ccf-a917-605a391b6394 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.357323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284cbb2-7883-4ccf-a917-605a391b6394 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.359154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 310/621] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:29.364747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.365035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.365750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.372819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 312/622] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:29.374296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.387258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:29.388404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:29.470438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 311/623] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:29.471601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.506639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623a5c8-ce5c-4e9c-aa2d-d1d6f263bd11 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.548019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.549142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.549602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.550091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.560845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.562629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.569944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:29.585322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:29.586255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:29.587515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:29.587515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:29.587515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:29.587515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:29.589738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 313/624] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:29.590977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.609408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fa21a-163b-4e06-8267-1d03bf04220a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.615312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.658158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.659220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.659502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.659777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.669896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.672258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:29.692653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:29.693759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:29.694721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:29.698447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 312/625] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:29.700168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.715440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ae7a4-7169-428c-8887-531d921d0e8a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.761204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.762360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.762743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.763429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.777031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:29.786699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:29.787841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:29.788674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:29.790395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 314/626] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:29.791377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.820077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c135d6-792c-4d89-bc33-1e2ba394bb24 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.858140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.865107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.866499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.867114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.867859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.868859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:29.869382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.872064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.887254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 313/627] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:29.889166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:29.905226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:29.926676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd01e70-11c1-4d29-857e-b680aab9c25a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:29.961963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:29.963390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:29.963767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:29.964213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:29.965320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:29.965825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:29.967554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:29.991669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 315/628] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:29.992586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.004950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0c8a0-e9b7-4ffa-91c3-5e34235c7b9b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.041739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.042794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.043274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.043738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.044695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:30.045264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.047659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.061937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 314/629] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:30.063304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.096973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cd9f6-ffd5-40a7-91b8-636919fafb4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.120878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.133665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.134508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.134870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.135060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.142701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.144298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.146077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:30.161568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:30.162614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:30.163502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:30.167964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 316/630] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:30.168910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.180511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b215b8-f857-4ef1-96d2-bd490d5c0916 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.217884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.218981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.219487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.219962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.277311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131eac94-ca06-4553-8a7b-723ab0263279 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.306057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.307891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036a895-f36b-4977-b45b-287d4da58591 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.313595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.314440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.314722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.314993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.327120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.328293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:30.346254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:30.347335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 317/631] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:30.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.375845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.403528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.431836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 315/632] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:30.431836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052db8-3c96-46f8-9cac-d6052532e918 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.431836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052db8-3c96-46f8-9cac-d6052532e918 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.431836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e052db8-3c96-46f8-9cac-d6052532e918 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.431836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 316/633] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:30.432275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e052db8-3c96-46f8-9cac-d6052532e918 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.466283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a971-a1a6-429b-80e4-693cfce01925 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.504719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.505446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.505729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.506028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.506850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:30.507162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.508611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.521223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 318/634] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:30.522094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.541598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e052db8-3c96-46f8-9cac-d6052532e918 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.575236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.575889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.576167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.576436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.577306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:30.577776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.579193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27974258-c8f3-45d3-88bc-7757be99cc14 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.591043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 317/635] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:30.592167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4939-6724-493a-8a96-93b9413d829d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.592514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4939-6724-493a-8a96-93b9413d829d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.592832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a4939-6724-493a-8a96-93b9413d829d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.611627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-100a4939-6724-493a-8a96-93b9413d829d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:30.612818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-100a4939-6724-493a-8a96-93b9413d829d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:30.620285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.624787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cd338-d3bf-4c05-a1c7-cddb4d4aa0ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.636025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.661852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.662899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.663180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.663449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.673509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.675103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.681459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 318/636] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:30.682796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc38ed2-39eb-457a-90f2-aa615fbc9b4d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.683078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc38ed2-39eb-457a-90f2-aa615fbc9b4d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.683351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc38ed2-39eb-457a-90f2-aa615fbc9b4d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.686382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 319/637] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:30.686382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc38ed2-39eb-457a-90f2-aa615fbc9b4d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:30.696983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:30.697868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:30.698791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:30.698791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:30.698791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:30.698791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:30.698791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 319/638] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:30.699451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.794066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc38ed2-39eb-457a-90f2-aa615fbc9b4d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.798030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdb2881-90ef-43ec-80a6-6616df536542 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:30.826662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.827640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.827920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.828192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2b2d6693e40d40f988254bbc7fcca45f/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.830618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:30.831413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.831690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.831960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.839611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.839825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.841068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.843099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=2b2d6693e40d40f988254bbc7fcca45f, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:30.845002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-74cefc94-b7df-4e38-a1f7-78ab73496b99 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:30.855975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:30.856921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:30.857902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 320/639] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:30.858509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 320/640] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:30 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2b2d6693e40d40f988254bbc7fcca45f/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:30.858579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:30.858654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.858722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:30.859460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:30.878483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:30.879827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:30.899876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:30.960791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 321/641] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:30.961914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:30.983052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f70be-c507-4d79-ad6b-b5ad2918edf9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.026896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.027851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.028129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.028398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.029122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:31.029423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.030887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.043745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 321/642] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:31.044894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.081112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dfb33-0057-44ad-9c73-48b5d75bc3df None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.105803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.118877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.119685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.119958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.120227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.131101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:31.138757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:31.140344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 322/643] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:31.141690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.156762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.165634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48bc21-cd19-4c79-a55f-3ac596a3048e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.204123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.204797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.205075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.205485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.206168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:31.206501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.207927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.226202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 322/644] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:31.227085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.251181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66a3b3-e311-45fb-818b-61732e92d6a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.287530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.288435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.288717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.288992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.300380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:31.308605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:31.309540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 323/645] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:31.311294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.337553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ccdf0-cd56-444f-9c59-3d6dbab5f339 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.349632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.379272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.380201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.380485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.380765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.383795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.390660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.392248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:31.408206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:31.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:31.409970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 323/646] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:31.410475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.420089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c425-0752-4d41-8c02-aecc7cd61e0d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.455782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.456455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.456847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.457126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.457879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:31.458185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.459725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.473658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 324/647] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:31.473658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.520476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cc13d8-6617-41db-a952-67025c6c0cec None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.563265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.563920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.564602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.564896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.578898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2793331-8774-4d31-b8ad-aa27d0429965 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.606653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.615897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.616575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.616851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.617131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.617887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:31.618197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.619620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.621079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.629455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 325/648] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:31.634456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.638519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.640010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.740735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6452a513-8ea9-4922-a79c-641fe451d769 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.751307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 324/649] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:31.751307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.779539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.780215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.780496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.780770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.781549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:31.781855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.783330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.801165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 326/650] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:31.801165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.827701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.857006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:31.859499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592a120-6074-42d9-b399-81676b1e153f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.894557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.895348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.895627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.895898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.903573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e785ac8-c74b-465d-bda4-9c9bc9480a92 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:31.906989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.908487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:31.926301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:31.927260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 325/651] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:31.928757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:31.947169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:31.947987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:31.948273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:31.948546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:31.961093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:31.962960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:31.981310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:31.982225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:31.983187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:31.983187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:31.983187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:31.983187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:31.987359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 327/652] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:31.987956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.040674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d58705-8e61-4469-9660-66cc45dacb82 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.068027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.082107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.082934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.083312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.083487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.094588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.096086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.105062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f816bf-3b4d-4eea-b029-1777693d270a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:32.114448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:32.115389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:32.116427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:32.116427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:32.116427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:32.116427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:32.117371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.118467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 326/653] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:32.119711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.144618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.145882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.146366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.146819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.147817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.148303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.149956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.163244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 328/654] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:31 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.164239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.239388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f740de0-8a9c-4060-9df0-75bcc671faa4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.293671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.294375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.294734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.295074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.296031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817272c-5b2d-416f-9b91-91812e7ad8f8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.296260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.296688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.298650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.318503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 327/655] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.319465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.346166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.346886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.347223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.347500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.348305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.349822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.351301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.363046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.370407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 329/656] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.371897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.406951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.441674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c81cd9-9316-4989-816a-b812628019eb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.481995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.482757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.483043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.483319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.484052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.484358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.485791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.490367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935cbcc-3c55-4f27-a68d-f1cd5ddad409 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.499384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 328/657] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.500254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.532194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.533316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.533749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.534180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.542059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.543850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:32.559426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:32.560637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 12:53:32.561835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:32.564050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 330/658] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:32.565013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.601956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.610758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e9c0c-d824-45a0-920f-a8d652cbce91 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.647886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.648691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.648969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.649241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.659405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.661324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.667512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.678190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc749710-8aaa-4101-9467-4d4399d0d812 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:32.680598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:32.681729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 12:53:32.682827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:32.686511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 329/659] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:32.690837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.722235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.723463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.723957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.724431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.731865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.733672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:32.748767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:32.749728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:32.751074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:32.757944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 331/660] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:32.759124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.811702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ba907-74d0-4927-896a-ada25dc8df71 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.858294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.859648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.860153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.860442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.861290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.861601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.864000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.865807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315882e-d04c-4ffc-ba72-20a13e37b4d0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:32.894513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 330/661] 10.222.0.49 () {56 vars in 1137 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.895700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.909229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:32.909907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:32.910434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:32.911126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:32.911248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:32.912013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:32.912365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:32.913924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:32.934104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 332/662] 10.222.0.49 () {56 vars in 1139 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:32.934763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:32.947785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.004951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa40fb7-5c02-472c-a78c-6b45bc3662f3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.040999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0fa9d-b73f-4e6e-b6c9-97f1a8a9097c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.041309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.041924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.042204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.042511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.043298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:33.043607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.045128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ae402-6b5e-4081-95ea-35655b5cf36c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 331/663] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a0d8-5e8c-4847-a976-6b2a3fb9f227 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a0d8-5e8c-4847-a976-6b2a3fb9f227 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79a0d8-5e8c-4847-a976-6b2a3fb9f227 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 332/664] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:33.062255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a0d8-5e8c-4847-a976-6b2a3fb9f227 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.085821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.086488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.086774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.087063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.139852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.154993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.156475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.167259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.174663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79a0d8-5e8c-4847-a976-6b2a3fb9f227 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.208668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.209310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.209590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.209857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.271346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 333/665] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:33.271346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.293655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.295355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b4767-863d-4c3f-954e-93170ae494d6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.367789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.370782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab0a48-b472-4d1b-8c50-8af6002eff94 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.382779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.409649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 333/666] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:33.409649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.409649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.409649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.410077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.410567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.410850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.411129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.412074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=607bae995849489db891582db395e28c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:33.412387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.413968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.427170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 334/667] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/role_assignments?user.id=607bae995849489db891582db395e28c&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:33.427170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.427471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:33.427471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:33.534557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 334/668] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:33.535623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.573061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc21383-4bde-4cab-ac16-2d76d599f664 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.613061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.613550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.613839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.614115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.630435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.632280 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:33.649731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:33.650674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:33.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:33.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:33.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:33.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:33.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 335/669] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:33.652337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.662656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94116497-d374-45bd-ab10-36c12b0103f6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.664674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.692419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.703961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.704933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.705285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.705626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.717293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.718981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:33.738360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:33.739445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:33.740524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:33.740524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:33.740524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:33.740524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:33.741857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 335/670] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:33.742675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.783273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb60710-be1b-40d9-bc2c-d9278d8f5461 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.819241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.820295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.820735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.821166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.832601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:33.841266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:33.842360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:33.843337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:33.850814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59537dc9-aa73-476a-a77e-917b5807e64b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.851206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 336/671] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:33.852373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.892766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.893447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.893821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.894166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:33.895021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:33.895414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:33.896971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:33.915007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 336/672] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:33.916033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:33.934071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.946313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:33.964070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88179b65-fbfa-4789-96f7-be66b1b70a43 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:33.998108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:33.998949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:33.999228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:33.999594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.000318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.000623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.002014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.014733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 337/673] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.015888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.029133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e92a34-631c-4db3-9bea-bb08db7ccb0c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.064570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.065533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.065988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.066440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.067538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.067995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.069520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.083484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 337/674] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.084761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.119731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad8fd8-4954-4522-bb56-76b3f709ef48 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.158093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.158839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.159118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.159389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.166850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.168312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.173411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:34.184450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:34.185452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:34.186459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:34.187287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 338/675] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:34.188138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.205174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.210312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f469e1-bd59-4a0a-babd-eba7dda4cd22 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.249384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.249884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.250169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.250464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.260488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.262256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:34.281067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:34.282019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:34.283358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 338/676] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:34.283838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.334898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211781f-3aae-4306-b5fa-156ef0dd9185 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.378622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.379969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.380757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.381508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.386088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe992fc6-892c-4444-b460-cd603616b4a6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.395533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.397960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:34.418467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:34.419376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:34.420345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:34.420345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:34.420345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:34.420345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:34.426853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.426853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.426853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.426853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.427196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.427266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 339/677] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:34.427342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.427417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.427497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176fc3d7-b083-47b7-a517-a84e533af703 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.429660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.440364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 339/678] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.441718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc18e-dd9e-4a4b-bd08-016a41d6bf52 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.442005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc18e-dd9e-4a4b-bd08-016a41d6bf52 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.442276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bc18e-dd9e-4a4b-bd08-016a41d6bf52 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.446363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 340/679] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:34.446363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc18e-dd9e-4a4b-bd08-016a41d6bf52 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.495934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.542014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db1aa4-135d-48a5-b74d-3489e8fe4138 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.551533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bc18e-dd9e-4a4b-bd08-016a41d6bf52 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.576129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.576818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.577133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.577450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.578237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.578579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.580023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.586879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.587895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.588173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.588447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.589199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.589505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.590910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6e7ca-9128-453b-9b07-7f9ea921f886 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.595424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 340/680] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.596569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.606387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 341/681] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.606387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.606387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.606387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.622788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:34.624042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:34.678048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.695398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 342/682] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:34.696075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.727412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428744e5-e52b-42ad-93f3-1d5de1189557 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.746804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.765679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.766354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.766677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.766938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.767714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=aaf234ea529642af8039f8e3b0bf601c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.768118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.769708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.782359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 341/683] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/role_assignments?user.id=aaf234ea529642af8039f8e3b0bf601c&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.782359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.808095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e9bfe-082a-4dc8-9a7f-a3d358acf1d1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.853997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.854848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.855189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.855526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.856450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=6e59c5b4dfe04f30a195ab3e83f49c2a) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:34.856821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.858474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:34.870825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 343/684] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/role_assignments?user.id=6e59c5b4dfe04f30a195ab3e83f49c2a&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:34.871923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:34.923692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cc388-57fd-4cd6-aa38-3b3edca67a19 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:34.971602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:34.972574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:34.972853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:34.973126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:34.975568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:34.986787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:34.989039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:35.009070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:35.010088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:35.011064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 342/685] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:35.019752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.048060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56155c-168a-4443-9dde-d66b11ad092a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.077916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.097567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.098905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.098905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.099100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.113406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:35.123713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:35.124826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:35.125624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 344/686] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:35.133912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.163755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a8cf4-fedf-43ea-8047-977ead5c49c2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.209890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.210907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.211198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.211714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.212919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:35.213228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.215437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.239409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 343/687] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:35.245236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.272043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfe606-2c09-42ac-98fe-04dcd609cbff None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.283778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.309435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.310109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.310414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.310690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.311622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:35.311898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.313282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.326725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 345/688] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:35.328115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.381977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.385626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d60d0-b0d0-46b8-80d5-9ae6104adc23 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.440078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.441523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.442555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.442835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.451499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48399aec-963e-4b52-868b-7b5d185c7965 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.507657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.508381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.508659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.508930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.533064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.534574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.564301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.565795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:35.567499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:35.568345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:35.569198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:35.569198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:35.569198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:35.569198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:35.580265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 346/689] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:35.587675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.642692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.689023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.693272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 344/690] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 12:53:35.699317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.745248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3190b6-98f9-442c-bd17-1574cdb7bab2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.782900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.783915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.784365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.784810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.785736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:35.786195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.787779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95816e3-371b-42fd-8b3f-e7d6fabcd44e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.801489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 347/691] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:35.807467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851a4-c5cf-4f5c-aeb1-41c73d1450ea None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.807997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851a4-c5cf-4f5c-aeb1-41c73d1450ea None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.808490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472851a4-c5cf-4f5c-aeb1-41c73d1450ea None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.810465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 348/692] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:35.815473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472851a4-c5cf-4f5c-aeb1-41c73d1450ea None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:35.823012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3928-be55-4068-a83b-83f1052d2146 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.857593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:35.858448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.858912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.858997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.869525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:35.871093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cb356196-45df-4086-bba6-6ff3e2dcd7cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:35.889319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:35.890752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:35.891643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:35.891643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:35.891643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:35.891643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:35.895196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 345/693] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:35.896474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:35.896758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:35.897030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:35.913732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:35.914957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:35.928488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:35.961505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472851a4-c5cf-4f5c-aeb1-41c73d1450ea None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:35.981097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.003786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.005002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.005485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.005957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.010445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 346/694] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:36.011961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.022119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.026097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:36.049088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:36.050059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:36.051042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:36.054445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 349/695] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:36.055433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.136082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd0f7f-6f6e-4f6f-a975-e9c19c03ebaf None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.172611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.174160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.174676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.175146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.176229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.176828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.178446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.178975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f952f98-9bc6-4f12-8e2b-292d5423e8ef None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.196494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 347/696] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:36.197997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.217834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.218680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.218959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.219230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.219892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.221474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:36.237308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:36.238685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:36.239588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:36.242354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 350/697] 10.222.0.49 () {56 vars in 1129 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:36.243427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.255497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.257301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.319064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f90903-f7e1-4b91-b9bb-c0fe12736373 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.355745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.356422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.356704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.356965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.357855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.358161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.359761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.362478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f79c2-827e-4b69-89e4-ad370dad7028 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.382268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 348/698] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:36.383179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.398635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc637-e407-455e-8f96-442edbee576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.399823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc637-e407-455e-8f96-442edbee576a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.400364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc637-e407-455e-8f96-442edbee576a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.400834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dc637-e407-455e-8f96-442edbee576a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.408605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298dc637-e407-455e-8f96-442edbee576a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.410352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298dc637-e407-455e-8f96-442edbee576a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-298dc637-e407-455e-8f96-442edbee576a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:36.425780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:36.426691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:36.427587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:36.434486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 351/699] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:36.435575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc637-e407-455e-8f96-442edbee576a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.481747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.486355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b21cd-45ee-4e2d-b91f-752b2575389f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.505883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.524464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.525413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.525863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.526302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.527213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.527688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.529289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.538914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 349/700] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:36.540049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.543394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dc637-e407-455e-8f96-442edbee576a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.579244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.580391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.580391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.580391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.646350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379e046-efcc-4249-ad8e-9e8c29fc8766 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.653211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.654822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.682902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.683945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.684420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.684886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.686213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.686774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.688516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.701827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 350/701] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:36.702991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.728799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.734806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:36.778370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 352/702] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:36.778370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.811506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11734f-f673-4ac9-a161-422533b3542f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.847265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.847998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.848367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.848732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.849537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.849943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.851447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.866488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 351/703] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:36.867673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:36.886241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c0230-32a3-454a-86b8-73cb12d9a87c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:36.927798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:36.928515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:36.928796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:36.929072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:36.929959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2b2d6693e40d40f988254bbc7fcca45f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:36.930379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:36.931869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:36.940134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 353/704] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/role_assignments?user.id=2b2d6693e40d40f988254bbc7fcca45f&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:37.017389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.039876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a0644-755f-4295-a500-cb377d7caf36 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.085419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.086228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.086552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.086822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.099777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.101245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-619cddff-5084-4d97-97c2-79293ab85812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:37.120275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:37.122362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:37.124533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:37.124533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:37.124533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:37.124533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:37.124533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 352/705] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:37.127598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.163633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.203556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b091940-a105-457f-a99e-dce9707d1853 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.204099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b091940-a105-457f-a99e-dce9707d1853 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.204099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b091940-a105-457f-a99e-dce9707d1853 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.205715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 353/706] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:37.211212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2df3-2dec-4fd0-b5cc-053809db84cb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.211612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2df3-2dec-4fd0-b5cc-053809db84cb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.211996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc2df3-2dec-4fd0-b5cc-053809db84cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.232221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aacc2df3-2dec-4fd0-b5cc-053809db84cb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:37.233610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aacc2df3-2dec-4fd0-b5cc-053809db84cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:37.251443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea46ca-327f-4a72-acc5-695ab9061b6a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.299295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.300543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.300543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.300543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.301792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:37.301792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.302988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.309031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 354/707] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:37.311267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d88f-90af-4d4c-b28d-aeb97facfe5e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.311267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d88f-90af-4d4c-b28d-aeb97facfe5e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.311267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99d88f-90af-4d4c-b28d-aeb97facfe5e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.312308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 355/708] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:37.316043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 354/709] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:37.319754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.320039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.320370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.325648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.348834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:37.350466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:37.433497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.440093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.445647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fd9a-f259-41b1-ad35-1b102889c10d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.450386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 356/710] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:37.451446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.498078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.500077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.500515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.501190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.509851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.512593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:37.534294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:37.535412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:37.536468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:37.539531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 355/711] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:37.540467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.589824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff133e-bff8-498c-a54f-18bb8be74ebd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.635007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295948-825f-488e-b652-343297c358dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.636515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295948-825f-488e-b652-343297c358dd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.636900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295948-825f-488e-b652-343297c358dd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.637291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295948-825f-488e-b652-343297c358dd None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.638063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295948-825f-488e-b652-343297c358dd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.640875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295948-825f-488e-b652-343297c358dd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-86295948-825f-488e-b652-343297c358dd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:37.659909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:37.661138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:37.662156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:37.669480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 357/712] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:37.669480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295948-825f-488e-b652-343297c358dd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.669715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bcd34f-ff80-4ace-ae4d-ca7836512577 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.710967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.711771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.712050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.712678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.729960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.731706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.749643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:37.751687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:37.752648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:37.753507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:37.753507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:37.753507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:37.753507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:37.757904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 356/713] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:37.758813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.763716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:37.798465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295948-825f-488e-b652-343297c358dd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.836587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.837974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.838673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.839175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.840166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:37.840802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.842862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.856829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 358/714] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:37.858127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.865188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a69e5-4c2e-4162-a3e1-72449c934b8a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.899633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:37.900223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:37.900501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:37.900778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:37.901492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] RBAC: Adding query filter params (name=network) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:37.901795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:37.903915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:37.913885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 357/715] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:37.913885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:37.974660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b2e98-012c-40fc-adc5-bd790c8dc52c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:37.981141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:38.002486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:38.014195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9042c81-1879-4200-8295-573b2519666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:38.015401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9042c81-1879-4200-8295-573b2519666f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:38.015950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9042c81-1879-4200-8295-573b2519666f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:38.016415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9042c81-1879-4200-8295-573b2519666f None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:38.017346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9042c81-1879-4200-8295-573b2519666f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:38.017841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9042c81-1879-4200-8295-573b2519666f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:38.019581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9042c81-1879-4200-8295-573b2519666f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:38.024505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35d437-a7e6-45ce-9608-1d467d5cf3fa None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:38.032117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 359/716] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:38.033241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9042c81-1879-4200-8295-573b2519666f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:38.066674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:38.067330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:38.067613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:38.067887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:38.068959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:38.069279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:38.071606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:38.079044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 358/717] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:37 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:38.080288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:38.150873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9042c81-1879-4200-8295-573b2519666f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:38.224855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:38.227422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:38.228046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:38.231903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:38.232969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] RBAC: Adding query filter params (type=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:38.233497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:38.235309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c478ba-5590-4d71-81d1-eeb3e278e1fd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:38.244549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 360/718] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:38.280492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a19437-aa9f-4d56-823d-d5b9ce74de4f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:38.327789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:38.335897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:38.337095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:38.337936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:38.338737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:38.339480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:38.340954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b83b2-ad1a-40a9-97f4-be9994f4d5cd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:38.388647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 359/719] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:53:38 2022] POST /identity/v3/users => generated 254 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:39.454783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27612a-ec2f-479c-9d48-682458e1c8c4 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.455546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27612a-ec2f-479c-9d48-682458e1c8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.455546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27612a-ec2f-479c-9d48-682458e1c8c4 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.456942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 361/720] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:39.462852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.463458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.463458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.491328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:39.492559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:39.504515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1d15-0a31-49d1-921b-5295e0a9518e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.504876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1d15-0a31-49d1-921b-5295e0a9518e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.505054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d1d15-0a31-49d1-921b-5295e0a9518e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.506703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 362/721] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:39.511299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.511572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.511819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.534618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:39.535836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:39.617332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 360/722] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:39.634992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:39.646586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:39.683620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 363/723] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:39.719331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:39.816478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0310c-7f01-426d-acdd-42f654231507 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:39.840480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:39.857296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:39.858713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.859220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.859689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.871296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:39.883720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:39.885323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:39.886159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 361/724] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:39.891892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:39.918149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654298-1408-4dc5-9bf2-aa56e9e0cf67 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:39.983563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:39.984238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:39.984512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:39.984786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:39.985520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:39.986932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:40.015422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 364/725] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:53:39 2022] POST /identity/v3/services => generated 251 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:53:40.054780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8572abf-5a05-404b-82be-faf2c6220a9b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:40.115790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:40.116542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.116831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.117117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.117885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:40.118194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:40.119685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:40.130131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 362/726] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:40.139802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:40.228454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:40.306847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3496f-056b-4996-b092-0db1e9236c7c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:40.347968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:40.348794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.349067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.349331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.369328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:40.370754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:40.390682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:40.391642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:40.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:40.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:40.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:40.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:40.392962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 365/727] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:40.397895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:40.442990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:40.524905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1da90-ae1b-4734-872c-f74ee42ab809 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:40.576959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:40.577626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.577901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.578171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.578931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:40.579233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:40.580596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94449416-1ac8-4ae5-b69b-ce8421439068 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:40.597134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 363/728] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:40.602393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:40.611944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbdd66-a1d4-4c5c-90fc-4e255eea4ae3 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.612240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbdd66-a1d4-4c5c-90fc-4e255eea4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.612486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbdd66-a1d4-4c5c-90fc-4e255eea4ae3 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.614194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 364/729] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:40.618970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.619400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.619621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.644947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:40.646252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:40.698501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:40.709815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440920e9-5cc5-4e5f-b776-30bb392d4671 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:40.767930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:40.768734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:40.769772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:40.770031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:40.783296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 365/730] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 12:53:40.784466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:40.786005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:40.800725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:40.801677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:40.803030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 366/731] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:40.807864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:40.818868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:40.850015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:40.952266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32672ec3-f3e5-4755-8072-9ad8a251350b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.002605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.003412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.003690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.003957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.018596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1f2bd-a4d4-4c26-bbd0-98de6577f8a5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.022972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fb7956a6-5055-41e9-8f36-1b5b217bae9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:41.030696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:41.031647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:41.032901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 367/732] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:41.033801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345fdf-549a-4c3f-92dd-d002ec4fcd98 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.033801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345fdf-549a-4c3f-92dd-d002ec4fcd98 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.033801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77345fdf-549a-4c3f-92dd-d002ec4fcd98 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.039174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 368/733] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:41.040017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345fdf-549a-4c3f-92dd-d002ec4fcd98 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.053881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.064821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.066018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.066496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.067151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.068741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:41.069302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.071585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24b2cd-cecf-4ca3-8dde-1c41c56df105 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.093318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 366/734] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:41.098200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.098200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.098200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.114117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:41.115693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:41.157779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77345fdf-549a-4c3f-92dd-d002ec4fcd98 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.192790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.193690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.194114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.194620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.195548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:41.196019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.197574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.206464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 367/735] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 12:53:41.207534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.213325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 369/736] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:41.222469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.237922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.299583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.329742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd87a8-750f-465b-87d7-1be608686862 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.332968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a97a4-f330-4d99-9ab1-5b62bc456172 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.367459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.368382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.368497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.368925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.369316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.369476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.369577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.369864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.386343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.386343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:41.401271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:41.402296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 370/737] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:41.403270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.448785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.452273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50187a1-8a0d-4e5f-bd3f-eed188dcb253 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.469615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.514990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5ff7-5845-495d-a222-3e9f1c08591c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.539336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.552518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.553201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.553484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.553756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.554522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.555919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.558358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 368/738] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 12:53:41.563896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ede-d6c6-490f-9975-3864fa22e69b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.564180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ede-d6c6-490f-9975-3864fa22e69b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.564450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef3ede-d6c6-490f-9975-3864fa22e69b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.565821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 369/739] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:41.566503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3ede-d6c6-490f-9975-3864fa22e69b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.582364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 371/740] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:53:41 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:41.582364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.703447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef3ede-d6c6-490f-9975-3864fa22e69b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.716063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ca6cb-4dc0-4185-bd36-7e9e3ada6315 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.721827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.756218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.760323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.760880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.761139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.761242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.761415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.761511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.761675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.762234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:41.762490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.764128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fec209-0c61-40ee-b188-7007f0fb8861 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.774820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.776284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.778067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 370/741] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:41.779825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.781097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.781097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:41.798608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:41.799537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:41.800403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:41.800403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:41.800403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:41.800403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:41.800403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 372/742] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:41.801296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.806132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:41.807616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:41.847032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:41.879281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 371/743] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:41.886340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:41.908785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658de5b-64b4-4bbd-91e3-a52a524895a3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:41.944845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:41.945699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:41.945977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:41.946246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:41.954976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:41.956797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:41.973534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:41.974515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:41.975463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 373/744] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:41.976034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.008712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.011928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410d4a6-9155-4da8-870a-ec33655d32c1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.047605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.048583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.049038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.049484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.050462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.050946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.052492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.064965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 372/745] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.066063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.078239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.085961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23118d-0b7f-4724-95dd-817a9f77d653 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.122666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.123624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.123984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.124362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.136358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:42.144494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:42.145433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:42.146252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:42.150850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 374/746] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:42.151956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.173995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb663f-a3f8-4c7c-a915-305cd7d36871 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.209478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.210542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.211046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.211539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.212924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.213442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.215098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.233860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 373/747] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.235093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.262724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.278900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab4591-fa0d-4c3c-8c93-36f6041d1b2e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.314612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.315413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.315697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.315960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.323110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.324669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:42.339678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:42.340634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:42.341530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 375/748] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:42.342520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.347060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b86ded-de62-4aca-be19-55b5905d6a60 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.361529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.384051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.385119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.385620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.386119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.387142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.387648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.389296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.398716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 374/749] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.399805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.448251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483fc43-25af-424d-bca1-ef71916ba959 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.484388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.484887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.486201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.486642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.486918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.498041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.499906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.503804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b8eae-d889-4c75-9606-42913e5d045f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:42.520026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:42.521085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:42.522000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:42.522000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:42.522000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:42.522000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:42.522000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 376/750] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:42.522310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.544562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.545640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.546059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.546515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.547314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.547619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.549265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07f5f2-894c-419b-8cad-0defa8d98d92 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 375/751] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1a99-a347-449f-83d9-d4e6e4bc0370 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1a99-a347-449f-83d9-d4e6e4bc0370 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc1a99-a347-449f-83d9-d4e6e4bc0370 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 376/752] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:42.567019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1a99-a347-449f-83d9-d4e6e4bc0370 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.600729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.638547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7725a5-8593-4696-b1e7-281e830370d5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.676018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.676822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.677090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.677343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.681028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc1a99-a347-449f-83d9-d4e6e4bc0370 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.688012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.689909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:42.708489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:42.709415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 377/753] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:42.710313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.725171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.725823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.726100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.726438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.727114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.727419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.728794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd901f4-0b66-4b11-8bfd-ea1ba65f2f2a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.738219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.742023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 377/754] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.743907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.744512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.745073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.765469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:42.767112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:42.818342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59749b04-4fb9-4063-a705-e2fd136cef85 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:42.843241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:42.854371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 378/755] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:42.854371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.875459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:42.876101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:42.876391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:42.876662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:42.877483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2b2d6693e40d40f988254bbc7fcca45f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:42.877796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:42.879285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:42.894388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 378/756] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/role_assignments?user.id=2b2d6693e40d40f988254bbc7fcca45f&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:42.894388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:42.971685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47df5-4835-4655-aa3a-3638bfb11735 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.011960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.019491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.020760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.021106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.021457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.022484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:43.022797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.024498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.032482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1426c8-8301-4cfc-814f-127cce21c272 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.041368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 379/757] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:43.042408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.074107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.074938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.075214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.075482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.082709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.084374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:43.099109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:43.100045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:43.101010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:43.105125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 379/758] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:43.105985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.152970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.180154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b2e61-2d35-4b80-a77c-257a77b808fc None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.211604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968995ad-a4e2-4876-a41e-4b11581a75cc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.222820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.224035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.224313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.224585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.225547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.228638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:43.244705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:43.245896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:43.246898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:43.250396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 380/759] 10.222.0.49 () {56 vars in 1129 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:43.251603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.262679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.263570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.264050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.264445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.275686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.277305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:43.293324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:43.294530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:43.295750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:43.297692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 380/760] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:43.298846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.323358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.377884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffbb51-16f7-4acd-bfe2-b56cbfc7ec50 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.412532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.413198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.413481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.413792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.414786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:43.415181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.416640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.419909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e243e26-0410-4101-9bd9-36c2fa9293ea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.434749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 381/761] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:43.435305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.435875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.464021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.465349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.465822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.466487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.467471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:43.467935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.470645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.480778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 381/762] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:43.480778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.552588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c01d7-17aa-4a96-9e3e-35e8cb14131a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.567416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.588176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf9fc-014f-4c5a-9ded-3acd132f6f86 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.589776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.590633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.591015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.591388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.592655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:43.593066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.594621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.613347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 382/763] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:43.614586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.625862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.627121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.627536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.627943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.630163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=2a0b0da6c54341ff8fb66f48a0a32b70, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.631958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.648974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 382/764] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:43 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:43.650157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.722796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.795297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162df0-3cfe-406d-bb10-df56c683c812 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.833391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bf7-7059-4551-981e-9779140c94ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:43.840942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.841607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.843872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.843872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.843872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:43.843872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.845418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05277192-5712-4594-b974-9a19565e9da3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.853079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 383/765] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:43.856293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca48e-c585-42c9-89aa-bd90b42b503e None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.856293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca48e-c585-42c9-89aa-bd90b42b503e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.856293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca48e-c585-42c9-89aa-bd90b42b503e None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.856883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 384/766] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:43.862649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.862786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.863241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.872849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:43.873820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:43.874392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:43.874838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:43.882695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:43.883915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:43.920359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:43.952452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:43.954297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:43.962785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 385/767] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:43.970795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:43.990681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.070714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 383/768] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:53:44.075891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.094542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b5a0-69dd-4a2b-921b-96c60e97c96d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.131154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.132122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.132688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.133132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.134167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] RBAC: Adding query filter params (interface=public, service_id=cfce2fcd5f8c4e6d8f61403280f01e4c, region_id=RegionOne) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:44.134693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.136574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.149865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 386/769] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:53:43 2022] GET /identity/v3/endpoints?service_id=cfce2fcd5f8c4e6d8f61403280f01e4c&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:44.151004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.175465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544300cd-9c35-4b4a-8f15-dc2c80d9dfe8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.190686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.203371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.211419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.212620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.213106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.213736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.214489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.215932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:44.228021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:44.229032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:44.229837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:44.235434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 384/770] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:44.236286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.264263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf4646-5a41-4b76-8ebb-9858e752fb6e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.302306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.303116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.303389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.303653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.318423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.319897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:44.337905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:44.338873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:44.339764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:44.339764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:44.339764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:44.339764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:44.342674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 387/771] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:44.343587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.355340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd9343-b063-4e99-9159-77c42831a088 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.398939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.399666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.399943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.400211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.400948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.402366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.414831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 385/772] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:44.415831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.449427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.493457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ae68f-0a21-41f4-9fdb-eee1bc7d6d62 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.545592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.555174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.555863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.556139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.556406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.557118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] RBAC: Adding query filter params (name=network) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:44.557428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.558837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.569700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 388/773] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:44.580232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.628808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e77097-f89c-4cb9-9c08-1f6ff0cb0e33 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.674268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.674968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.675848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.676134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.676904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:44.677211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.679077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.693350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 386/774] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:44.701074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.723496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886474c-10ea-4147-ad5a-79b0cf8d0e41 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.791932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.792598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.793651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.793927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.794662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:44.795016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.796423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:44.807883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 389/775] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:44.815845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:44.846709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.905637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdee3b5-3c2b-40ec-8cac-918a19bf80ba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.935898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:44.977826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:44.978652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c3e35-8bde-4585-8803-c980d11faaf9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:44.978795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:44.978978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:44.979201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:44.995653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:44.997159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.014079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.014995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.015287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.015568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.016372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] RBAC: Adding query filter params (interface=public, service_id=7a08e0f0b2154260ad2fafceda8571dc, region_id=RegionOne) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:45.016685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.018158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:45.019400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:45.020259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:45.021137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:45.024309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 390/776] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/endpoints?service_id=7a08e0f0b2154260ad2fafceda8571dc&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:45.028074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 387/777] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:45.029782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.029919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.111915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:45.151019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63ebd8-14e1-4eb9-93b8-db33bdadee3c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.152455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a63d92-9da5-4c5b-a9f9-e688a997b471 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.186952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.187700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.188070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.188439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.189319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:45.189718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.191531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.192824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.193905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.194585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.194876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.195566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.196911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3938a22-e843-474b-a817-8f6da38dbae3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.203591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 391/778] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:45.205727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06767c1e-b662-4656-8ebc-5c4ac68ae00c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.206222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06767c1e-b662-4656-8ebc-5c4ac68ae00c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.206825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06767c1e-b662-4656-8ebc-5c4ac68ae00c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.213266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 392/779] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:45.215308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.215819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.216073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 388/780] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:45.216521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.226706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.244165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:45.245864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:45.330524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 393/781] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:45.342502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:45.354762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.426044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:45.463189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0e4b0-be3e-4606-bf62-9e31f3bf8906 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.477616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5805b92e-e394-43bd-bd37-9944d9551015 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.509334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.510566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.511044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.511519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.528671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:45.533254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.534161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.534679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.535145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.536163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=1bc929bde94d48e2b48e959d970ab1d1) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:45.536358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:45.537207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:45.538016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 394/782] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:45.539609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.541263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.543831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.549855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 389/783] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/role_assignments?user.id=1bc929bde94d48e2b48e959d970ab1d1&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:45.653200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ec31e-4916-4979-a69d-679c7123b385 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.682759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:45.691358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.692386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.693040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.693312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.694141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:45.694480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.695967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73878c-7e8b-4025-9862-26f02fb1a1b1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.706104 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 395/784] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:45.711668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:45.830043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5869-7d74-48bf-bfb4-c2d6b359dec5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.830220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5869-7d74-48bf-bfb4-c2d6b359dec5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.830518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed5869-7d74-48bf-bfb4-c2d6b359dec5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.832066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 396/785] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:45.837745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.838051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.838296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.856172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:45.857470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:45.866880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:45.871752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7744e-81fb-4c5d-a936-bcf8a4d9550c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:45.909608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:45.910467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:45.910752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:45.911028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:45.925651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:45.927183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:45.952416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 397/786] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:46.019140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:46.104430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:46.105338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:46.106223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:46.106223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:46.106223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:46.106223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:46.107714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 390/787] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:46.112104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.249626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269da6f5-fa09-44b0-8643-ac4d36695a81 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.278451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3df73-1b61-4490-97b9-a6e474d42623 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.282255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:46.313827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.314833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.315107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.315378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.316158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.316832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.317109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.317379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.318104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:46.318499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:46.319902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:46.332427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 391/788] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:46.339276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.347578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:46.360421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:46.361346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:46.362168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:46.366349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 398/789] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:46.369030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.374153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:46.508032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e20dc-6399-423b-8dd1-bbe237bda2b4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.541922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b21ef-4c39-46aa-979a-bb84446d5c44 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.579332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.580429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.580912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.581514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.593062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:46.595667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:46.608401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.609343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.609817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.610280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.611380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:46.611894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:46.613560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:46.622191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:46.623396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:46.624476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:46.628252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 392/790] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:46.633696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.640232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 399/791] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:46.643707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.743790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:46.754704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:46.808082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c852782-2c86-4e39-9d9e-d13ea359d257 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.846117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9286-900d-4ad1-a4ae-33e2564d0034 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:46.856990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.858308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.858845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.859550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.861046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:46.861865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:46.864289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e214ba3-c591-4e21-8fb7-f5c9355f91a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:46.882503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:46.883307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.883586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.883857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.890376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 393/792] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:46.890376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd124-b87f-4bdd-a930-05378e29ccc8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.890376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd124-b87f-4bdd-a930-05378e29ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.890376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd124-b87f-4bdd-a930-05378e29ccc8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.890376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 394/793] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:46.892319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd124-b87f-4bdd-a930-05378e29ccc8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:46.906276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:46.907780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-4807e7c4-dfd1-44a7-a0ae-aa6c8d00a9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:46.926446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:46.927410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:46.928783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:46.928783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:46.928783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:46.928783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:46.928783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 400/794] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:46.929313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:46.934276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:46.934619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:46.953298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:46.954519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:47.021165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affd124-b87f-4bdd-a930-05378e29ccc8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.026167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 401/795] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:47.027342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.052999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.065484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.073136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.074162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.074664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.075106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.086962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.089510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:47.112722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:47.113632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:47.114546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:47.114546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:47.114546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:47.114546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.116384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 395/796] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:47.117351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.142053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc19f-e3ea-406c-9510-bb034f9bcb36 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.177110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.177925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.178302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.178695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.179537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:47.179943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.181462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.198055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 402/797] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:47.198055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.244147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfadba8f-df3b-4703-b8ee-21498c8f2614 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.287656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.298146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.299860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.300314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.301113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.302143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.304378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.305832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e4d25-c975-473a-a64a-600a3e40ad87 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:47.320740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:47.321872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 12:53:47.322939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.330870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 396/798] 10.222.0.49 () {56 vars in 1129 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:47.330870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.355851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.356894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.357445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.358505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.359716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:47.360230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.362589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.379487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 403/799] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:47.380503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.389093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.452078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa533f-f876-4ebd-a4b9-d4483c11df4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.501630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819bb35-43a1-4dca-abee-b56b21780557 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.507645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.508655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.509096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.509614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.521356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.523055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.538817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.539637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.540021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.540425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.541232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:47.541659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:47.542559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:47.543628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:53:47.544776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.545971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.552530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 397/800] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:47.552530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.555779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 404/801] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:47.557242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.575086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.670647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33dfaeb-b800-46b0-a7d5-7ebbacd14d8b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.681913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b653e7-af85-4cd4-8aad-d314ac30549f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.690087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.709405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.710075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.710464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.710768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.711609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:47.711915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.713696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.718940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.719730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.720004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.720268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.727098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.728540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.736348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 405/802] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:47.736348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2b36f9f7-5de5-4e52-9ffe-4feb0109a032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:47.745042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:47.745885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:47.746796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 398/803] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:47.747702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221d54-9e33-47f0-9403-9c8cf2dd8494 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.747702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221d54-9e33-47f0-9403-9c8cf2dd8494 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.748235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94221d54-9e33-47f0-9403-9c8cf2dd8494 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.753554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 399/804] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:47.753554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94221d54-9e33-47f0-9403-9c8cf2dd8494 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.805851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.843776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809091e2-b295-4a8d-9e12-489efb4638b6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.856437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94221d54-9e33-47f0-9403-9c8cf2dd8494 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:47.880168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.880830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.881104 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.881370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.882066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:47.882394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.883750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.888865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 406/805] 10.222.0.49 () {56 vars in 1124 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:47.889556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:47.895911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:47.896574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:47.896851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:47.897119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:47.931823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:47.968668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:47.970146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:47.988478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a346f6-9b4a-49af-b9c4-b76b4790d44a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.015951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.021974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.022657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.022941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.023216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.024003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:48.024311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.025730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981b0f3-ce22-4161-82d6-586a7a8e0d8f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.037162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 407/806] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:48.038365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81184-f942-4530-879f-1e7e66537652 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.038649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81184-f942-4530-879f-1e7e66537652 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.038924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e81184-f942-4530-879f-1e7e66537652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.054809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c9e81184-f942-4530-879f-1e7e66537652 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:48.055984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c9e81184-f942-4530-879f-1e7e66537652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:48.066558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 400/807] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:53:48.067377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.119903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 408/808] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:48.121227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6f67d-3358-48f2-baec-26545b60a822 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.121541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6f67d-3358-48f2-baec-26545b60a822 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.121777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6f67d-3358-48f2-baec-26545b60a822 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.123141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 409/809] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:48.123920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6f67d-3358-48f2-baec-26545b60a822 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.140084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.168381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c7be4-5e5a-405b-a15f-7e545f6cafbd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.200573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.201214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.201683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.201958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.202731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.204102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.225871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 401/810] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:53:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:48.226722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.231661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6f67d-3358-48f2-baec-26545b60a822 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.250794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.271932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.272838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.273114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.273554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2b2d6693e40d40f988254bbc7fcca45f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.275491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=2b2d6693e40d40f988254bbc7fcca45f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.276958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b460e0-dc8f-4d3a-a258-406b9d920c41 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.294497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 410/811] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:48 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2b2d6693e40d40f988254bbc7fcca45f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:48.296654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.296942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.297215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.327023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:48.328377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:48.374522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ea512-c481-4762-9045-1cb6af135228 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.405035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.423876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.425190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.425190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.425388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.431207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 411/812] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:48.431872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.439808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.442027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:48.469994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:48.471035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:48.471901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:48.471901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:48.471901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:48.471901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:48.473852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 402/813] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:48.474940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.549503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c39c83-7b0f-4b61-93df-d30b25fbd2d9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.557852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.581704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db6e1d-b539-4705-8787-0c2ff485c65a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.589325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.590503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.590987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.591524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] PATH_INFO: `/v3/services/network` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.592396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.594047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:48.606972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:48.608075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 12:53:48.608893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:48.611936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 412/814] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:48.612941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.616811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.618482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.618767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.619045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.632774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:48.643045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:48.643988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:48.644823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 403/815] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:48.645500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.691041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.739370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac1005b-a54a-4bc7-a6bc-1f39c9d4b67a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.759469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f68f3-3581-4113-aa06-8328513e4aaa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.779085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.780215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.780215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.780504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.781234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:48.781541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.783858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.796815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.797834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.798308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.798946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.799844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] RBAC: Adding query filter params (name=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:48.800356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.801938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.804219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 413/816] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:48.805714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.809142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 404/817] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:48.810267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:48.845152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.937655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf67bfc-ac95-4540-a7f6-b01a8a633c8e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.940224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:48.945088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f57b2d-22dc-4aec-8d9e-77d0d1415485 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:48.977511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.978380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.978757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.979123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.980101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:48.980501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:48.982093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:48.990510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:48.991518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:48.991823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 414/818] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:48.992154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:48.992638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:48.992969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.002007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.004296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:49.024965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:49.026084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:49.027196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:49.030108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 405/819] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:49.031130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.106284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.118345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f84adb-7fc7-4004-aba1-4a0571b2c6f2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.143432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96cef2-4218-47d5-87a7-f21e691f1ad9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.160732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.161733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.162460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.162902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.163756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] RBAC: Adding query filter params (type=network) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:49.164204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.166120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.178592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 415/820] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:53:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:49.178592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.190941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.192374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.192812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.193236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.207967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.210135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:49.234714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:49.235899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:49.237006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:49.237006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:49.237006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:49.237006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:49.239001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 406/821] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:49.240342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.257778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.295127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526ce8-d677-4624-b53f-bab100920f67 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.330534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.331751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.331751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.332062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.332935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:49.333379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.334935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.353272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 416/822] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:49.354269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.365130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ae1af-3630-4262-8d07-bb34e659565d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.372635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.402955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.403931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.404441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.405497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.406486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.408107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.438539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 407/823] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:53:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:49.439934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.464313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fbd25-b106-498a-913b-5c99f5cbdc2d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.526956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.527595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.527874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.528146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.528878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:49.529211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.532821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.557044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 417/824] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:49.567202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.577432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.616919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4524a7-2af8-4fa0-b2eb-b33709cd1e04 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.681332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.681995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.682275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.682571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.683405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:49.683711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.686394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.691758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.693573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 408/825] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:49.729593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447505e1-eee7-4433-9742-bf9674073425 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.766907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.767743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.768022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.768296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.777211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.778617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:49.794479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:49.795561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:49.796469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 418/826] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:49.802547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:49.921558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48447bde-ce2c-4901-98a8-3f284c0c6164 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:49.951885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:49.958300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:49.959001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:49.959278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:49.959546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:49.960381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:49.960686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:49.962115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:49.981471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 409/827] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:49.987912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.100225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dceae-783c-4055-b08b-96f4cb205951 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.132035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.138049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.139289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.139879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.140375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.152008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.153780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:50.172421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:50.173347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:50.174234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:50.174234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:50.174234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:50.174234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:50.176646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 419/828] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:50.183451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.289600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d5d75-06d8-4b36-b81c-b7b2c16f4990 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.325713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.326868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.327365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.327870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.328823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:50.329357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.330980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f984f-cf2c-4ce2-8e09-9ebf60030a16 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.344117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 410/829] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:50.350590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.392620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d8f0-6c10-4181-91f6-74e81c03f789 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.392946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d8f0-6c10-4181-91f6-74e81c03f789 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.393186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5d8f0-6c10-4181-91f6-74e81c03f789 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.394893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 411/830] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:50.400048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5dc1d-f975-427b-8e37-dffe67703eb5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.400203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5dc1d-f975-427b-8e37-dffe67703eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.400419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5dc1d-f975-427b-8e37-dffe67703eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.417944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-55e5dc1d-f975-427b-8e37-dffe67703eb5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:50.419167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-55e5dc1d-f975-427b-8e37-dffe67703eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:50.465263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2dc73b-c234-441b-a686-d4ecda595f31 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.472765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.502434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.503241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.503520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.503794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.504571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 412/831] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:50.506095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea380f-a2df-4740-9df8-62b19296c02a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.506614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea380f-a2df-4740-9df8-62b19296c02a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.507203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea380f-a2df-4740-9df8-62b19296c02a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.508642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 413/832] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:50.513942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.514983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.515573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.519604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.521405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.534285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:50.535652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:50.537513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:50.538794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 420/833] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:50.539883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.592402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.608884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 414/834] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:50.609541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.661896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534be5d-3e7b-4c3c-924d-db66b2c7b5d1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.697162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.697992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.698275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.698593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.699247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.701067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.712632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346b824-df77-488f-bf84-b5319586cfd3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:50.713244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:50.714176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:50.715081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 421/835] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:50.715988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.727390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.755843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853ea1-8204-4341-a681-738805857347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.756529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853ea1-8204-4341-a681-738805857347 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.756805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853ea1-8204-4341-a681-738805857347 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.757078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f853ea1-8204-4341-a681-738805857347 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.757885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853ea1-8204-4341-a681-738805857347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:50.758191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853ea1-8204-4341-a681-738805857347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.759698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f853ea1-8204-4341-a681-738805857347 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.778474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 415/836] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:50.778474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853ea1-8204-4341-a681-738805857347 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.887782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f853ea1-8204-4341-a681-738805857347 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.924920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.925950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.926231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.926788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.927499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:50.927963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:50.929630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:50.940340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 416/837] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:50.940340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:50.943538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05f821-3ec0-456a-8b5d-2fbd59c0c41a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:50.953348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.975299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:50.982875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:50.983684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:50.983959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:50.984224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:50.996754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:51.004603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:51.005600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:51.006940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 422/838] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:51.007616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.064938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7db7c-06d0-4dbd-9f1d-40ba1ff9127f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.102585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.103631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.103914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.104188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/1bc929bde94d48e2b48e959d970ab1d1/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.113575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd049e-8652-4600-8d06-307a750379bf None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.119936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=1bc929bde94d48e2b48e959d970ab1d1, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.121812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.146453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 417/839] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:53:50 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/1bc929bde94d48e2b48e959d970ab1d1/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:53:51.146453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.158486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.159243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.159521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.159793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.160509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:51.160815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.162413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.179616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 423/840] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:51.246100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:51.277299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6a3f5-2ad3-4d4d-8195-f0db58ab3b96 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.329927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.330685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.330968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.331242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.332023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:51.332333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.333781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.345252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 418/841] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:51.350520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.385498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:51.466735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd4cca8-54f6-48bd-b9d0-e83e4532d127 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.502447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.503253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.503535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.503810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.516064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.517536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:51.543391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:51.544348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:51.545466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:51.545466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:51.545466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:51.545466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:51.545466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 424/842] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:51.552053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.660905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481a1d5-67c5-4366-802f-bcff8e91bd43 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.697275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.698361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.698845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.699334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.700278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:51.700793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.702441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.716177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 419/843] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:51.722233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.807312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:51.823675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6944-a053-4a8b-a202-975dd4ce1d69 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:51.858222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:51.859047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.859323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.859594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.867540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:51.869030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b2bb0919-ee37-4fcc-b881-f73735d2e0ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:51.884065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:51.885058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:51.885995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 425/844] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:51.893090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:51.897577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:51.932722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec01e-347c-4e83-9b7a-fcdc9b9f730b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.933058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec01e-347c-4e83-9b7a-fcdc9b9f730b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.933280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ec01e-347c-4e83-9b7a-fcdc9b9f730b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.934953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 426/845] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:51.942629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:51.942912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:51.943187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:51.960872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:51.962088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:52.003260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaf55d-5c14-44de-8275-ae2cf8c88652 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.038423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 427/846] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:52.046022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.046862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.047143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.047460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.048364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:52.048676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.050123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.056938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.068862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 420/847] 10.222.0.49 () {56 vars in 1135 bytes} [Thu Jun 30 12:53:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:52.074338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.134439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.140297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.162576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952ce5b-a826-46da-84e2-9a3c5cb3881d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.181444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7bc66-1ccd-4081-b0e2-d03671267a1e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.199202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.200108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.200387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.200653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.202029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.203248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:52.216612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:52.217874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:52.218908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:52.222953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 428/848] 10.222.0.49 () {56 vars in 1132 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:52.223555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.224206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.224678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.225134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.225551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.311511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.313070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.336395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cf810-021a-4070-8bb1-f593c21c4793 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.377607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.378550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.378899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.379491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.380460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:52.380769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.382881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.389280 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.389427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 429/849] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:52.395508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.400051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.432131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 421/850] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:53:52.439291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.523197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e6d8d-f348-4c2b-974a-64e07d4c7701 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.557636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af73c56-f3f5-4916-9502-06bdb495b649 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.558216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.559061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.559524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.559974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.561184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:52.561817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.563449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.568805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 430/851] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:52.597266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.598089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.598413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.598696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.610491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.611978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:52.631217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:52.632171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:52.633137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:52.633137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:52.633137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:52.633137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:52.633137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 422/852] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:52.639899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.713960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.755429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63c13b-1049-4347-adb7-f673faf11206 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.792609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.793196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.793475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.793747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.794493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:52.794853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.796228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.809190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 431/853] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:52.815124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:52.831665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:52.919878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec53ec-f2bf-4f61-97bc-d85c926d57a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:52.955309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:52.956115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:52.956399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:52.956672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:52.964392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:52.965866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:52.980455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:52.981377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:52.982739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 423/854] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:52.990054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:53.095060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7831c6f-660d-49c2-9ba5-817b70236e13 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:53.123970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:53.134784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:53.135753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.136235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.136744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.137717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:53.138219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:53.140149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48c00f-79c4-4b87-bbf3-89356c4a7a99 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:53.153093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 432/855] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:53.158964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:53.234701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d782a33-a7fe-40f1-88ff-ab36396f60a1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.235338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d782a33-a7fe-40f1-88ff-ab36396f60a1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.235808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d782a33-a7fe-40f1-88ff-ab36396f60a1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.237655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 433/856] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:53.240972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:53.242697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec341-e8c6-48ec-a87a-20bbe69fa747 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.242993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec341-e8c6-48ec-a87a-20bbe69fa747 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.243276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec341-e8c6-48ec-a87a-20bbe69fa747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.265953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2e5d5-f18f-45ba-99ac-d0deeb1f63ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:53.271311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-451ec341-e8c6-48ec-a87a-20bbe69fa747 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:53.272745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-451ec341-e8c6-48ec-a87a-20bbe69fa747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:53.300826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:53.301404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.301687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.301961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.302828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2b2d6693e40d40f988254bbc7fcca45f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:53.303141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:53.304722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:53.312060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 424/857] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:53 2022] GET /identity/v3/role_assignments?user.id=2b2d6693e40d40f988254bbc7fcca45f&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:53.352078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 434/858] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:53.374493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:53.426208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84bea-5578-49cf-ae0b-168c933f14e1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.426581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84bea-5578-49cf-ae0b-168c933f14e1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.426790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d84bea-5578-49cf-ae0b-168c933f14e1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.428307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 435/859] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:53.434763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.435156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.435320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.458284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:53.460921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:53.496558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8d0e6-8b1c-4caa-82ee-136fb6efffb8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:53.542775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:53.543565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.543849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.544108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.545097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:53.546852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:53.551177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 436/860] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:53.564709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:53.574623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 425/861] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:53:53 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:53.579585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:53.634413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:53.699078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5922-a884-4f2f-93a9-2448ef255342 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:53.745308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:53.746126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.746434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.746706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.758529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:53.766570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:53.767473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:53.768276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 437/862] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:53:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:53.773720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:53.879983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a73f3f-eae9-4caa-b864-8ac37e8cee46 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:53.914090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:53.914703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:53.914981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:53.915251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:53.916001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:53.916303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:53.917679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:53.926452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 426/863] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:53:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:53.931952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.035129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4b7d2-dda3-42c6-99db-eb4ace5ae348 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.036666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:54.071097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:54.071927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.072210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.072484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.083754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:54.085210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:54.103158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:54.104573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:54.105513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:54.105513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:54.105513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:54.105513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:54.105513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 438/864] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:54.111723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.175384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:54.217121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65102673-48d7-4f82-8d63-2364a027fe36 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.251201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:54.251926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.252204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.252478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.253245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:54.253556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:54.254962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:54.267511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 427/865] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:54.275443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.380967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c8eed-729c-456d-82b2-ae503f5db524 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.416439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:54.417629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.418120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.418643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.426940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:54.428614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:54.443737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:54.444693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:54.445612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:54.447910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 439/866] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:53:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:54.453405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.560096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9da54-cc4b-461b-a9d4-ce50e65f0651 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.580884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:54.603859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:54.604795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.605228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.605639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.606724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:54.607174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:54.608816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838f454-8db2-4373-93f8-bc8558e50592 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:54.627159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 428/867] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:53:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:54.632636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.650764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:54.715359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682592-d396-47ff-85dd-e95066dcb28c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.715555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682592-d396-47ff-85dd-e95066dcb28c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.715828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2682592-d396-47ff-85dd-e95066dcb28c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.718157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 429/868] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:54.722866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.723152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.723538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.741580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:54.742799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:54.743933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313a37a-d212-4fb4-a35b-5ed1942dab7d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.778770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:54.779581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:54.779955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:54.780230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:54.814760 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 430/869] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:54.841839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.862245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:54.863729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:54.880649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:54.972310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 440/870] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:54 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:53:54.975862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc96078-e0a9-409a-885f-bf9aa18e6968 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:54.979393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:54.988534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.021249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.021924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.022202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.022495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.023227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.025132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.044446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 431/871] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:53:54 2022] POST /identity/v3/services => generated 269 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:53:55.100155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887c73-d04a-42ae-9079-5ac71061b410 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.117337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.149842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.151215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.151718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.152226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.164832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.166771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:55.187425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:55.188384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:55.189269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:55.189269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:55.189269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:55.189269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:55.191748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 441/872] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:55.197549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.311645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e553dd-2975-473f-a68a-8f5470cb3cba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.346922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.347960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.348452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.348916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.349858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:55.350374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.351963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be0142-aa05-4730-bb62-fb4c4014608a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.365269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 432/873] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:55.371066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.465607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5d5f-28f3-4e37-95fe-4934447864bd None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.466125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5d5f-28f3-4e37-95fe-4934447864bd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.466125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a5d5f-28f3-4e37-95fe-4934447864bd None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.467851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 433/874] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:55.472892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.473170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.473447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.479612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3d51e-2b9f-4f54-a4c6-55ea10d9c011 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.490174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:55.491659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:55.513498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.514272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.514582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.514857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.522007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.523479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.523594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:53:55.538695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:55.539628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:55.540529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 442/875] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:55.547337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.586703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.591191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 434/876] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:55.615416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.663305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479f875-ddd7-457b-9128-3f6a69486a21 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.705229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.705890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.706166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.706467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.707250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:55.708060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.709780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.720931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8aa1ea-6afe-41a3-bc8b-1fc6b99f817d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.727392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 443/877] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:55.735249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.765991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.766921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.767204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.767477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.768490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.770165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3085fbe7-0719-4494-8ff5-1d5146bb2183 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:55.783473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:55.784591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:55.785710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 435/878] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:55.786684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c380-b788-4f68-87ee-06534adf00a9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.786684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c380-b788-4f68-87ee-06534adf00a9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.786684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c380-b788-4f68-87ee-06534adf00a9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.787927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 436/879] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:55.791153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96c380-b788-4f68-87ee-06534adf00a9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:55.818584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.831237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:55.850912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62440af-8705-471c-8522-151cd29eb5bc None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.884658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.885091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.885368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.885638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.886563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=1bc929bde94d48e2b48e959d970ab1d1) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:55.886870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.888352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658d80d-0df4-49ea-9bb6-1ba2ed0e60d9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.894538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 444/880] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/role_assignments?user.id=1bc929bde94d48e2b48e959d970ab1d1&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:55.895719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.896004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.896274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.914466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:55.915692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:55.921315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96c380-b788-4f68-87ee-06534adf00a9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:55.973177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:55.974583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:55.975091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:55.975811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:55.977181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:55.977680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:55.979322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:55.988045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 437/881] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:55.993132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.000888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 445/882] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:56.024759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.097974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:56.117020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:56.119746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416aa97-e19d-4809-a6c5-115b62a228e2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.151822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3daa6dc-e302-446b-9c75-8136fe95c45d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.164402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.165352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.165847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.166119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.166950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:56.167255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.168645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.173630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 438/883] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:53:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:56.193677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.194582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.194947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.195224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.195895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.197443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:56.211197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:56.212591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:56.213397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 446/884] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:56.219481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.330912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb795-2359-4749-8fa2-228514eb9b9e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.367829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853295b-238b-4254-9589-1f490c863563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.368513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853295b-238b-4254-9589-1f490c863563 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.368790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853295b-238b-4254-9589-1f490c863563 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.369063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853295b-238b-4254-9589-1f490c863563 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.369783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853295b-238b-4254-9589-1f490c863563 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:56.370090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853295b-238b-4254-9589-1f490c863563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.371585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853295b-238b-4254-9589-1f490c863563 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.377836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 439/885] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:56.383555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.484976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077faeb6-8be9-4e44-8af3-f5c2f1d42ea1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.520939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.521590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.521866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.522135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.522872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:56.523173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.524529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.530129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 447/886] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:56.535893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853295b-238b-4254-9589-1f490c863563 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.552561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:56.647053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853295b-238b-4254-9589-1f490c863563 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.681757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.682444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.682727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.682997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.683790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] RBAC: Adding query filter params (interface=public, service_id=fd582cafc96545689a43f0748d824e96, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:56.684099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.685499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3fbf-de6e-4bb5-8e65-083f32b7c177 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.693598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 440/887] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:53:56 2022] GET /identity/v3/endpoints?service_id=fd582cafc96545689a43f0748d824e96&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:56.699445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:56.704629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:56.812833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276daad-4ce5-483f-a731-216d65c1f00a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:56.847614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:56.848277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:56.848554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:56.848823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:56.849556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:56.850962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58efaf-8ba7-427f-87d4-c4eee785b6b9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:56.857124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 448/888] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:53:56 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:53:57.201194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5a911-2265-454d-8ee4-684cb76bc2e3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.201590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5a911-2265-454d-8ee4-684cb76bc2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.201809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5a911-2265-454d-8ee4-684cb76bc2e3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.203493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 441/889] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:57.208747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.209203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.209278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.227076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:57.228258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:57.303824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:57.310010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 449/890] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:57.332841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5a911-2265-454d-8ee4-684cb76bc2e3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:57.414697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:57.441793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5a911-2265-454d-8ee4-684cb76bc2e3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:57.476808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:57.477887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.478348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.478813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.479626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:57.481144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:57.493722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:57.494876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:53:57.495899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:57.498561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 442/891] 10.222.0.49 () {56 vars in 1132 bytes} [Thu Jun 30 12:53:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:57.506381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:57.609503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db78fd-1801-43d1-b441-e24cc41d13e5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:57.643892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:57.644552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.644828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.645100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.645804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:57.646103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:57.647494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:57.652576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 450/892] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:53:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:57.658177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:57.763592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea24403-e75f-4982-8180-3c1339182402 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:57.797035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:57.797699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.797974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.798241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.798973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:57.799275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:57.800650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:57.805886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 443/893] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:53:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:57.811698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:57.875379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:57.915987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b6fe95-befb-4112-a92d-97c58c61fc43 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:57.950875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:57.951599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:57.951935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:57.952207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:57.953046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] RBAC: Adding query filter params (interface=public, service_id=654ab64024b2478c949072da48ce5dd4, region_id=RegionOne) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:57.953361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:57.954813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e8a24-a217-4668-81b4-1e9dce3d2f00 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:57.960719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 451/894] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:53:57 2022] GET /identity/v3/endpoints?service_id=654ab64024b2478c949072da48ce5dd4&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:57.967167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:57.977062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:58.076209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfedc0-564e-4cae-a76b-4004ed8b6090 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:58.110533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:58.111204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.111481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.111750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.112432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:58.113859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a768e-7c38-4b6b-8e1d-75b4f06e7cda None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:58.120571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 444/895] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:53:57 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:53:58.165658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c3f7-f589-4360-a1ad-62c4e8323bd1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.165658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c3f7-f589-4360-a1ad-62c4e8323bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.165658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42c3f7-f589-4360-a1ad-62c4e8323bd1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.166383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 452/896] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:58.172351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ce4c-f71b-4aff-8cb0-febcaf846877 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.172878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ce4c-f71b-4aff-8cb0-febcaf846877 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.173294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ce4c-f71b-4aff-8cb0-febcaf846877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.191317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b154ce4c-f71b-4aff-8cb0-febcaf846877 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:58.192799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b154ce4c-f71b-4aff-8cb0-febcaf846877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:58.272080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:58.274398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 445/897] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:58.292820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c3f7-f589-4360-a1ad-62c4e8323bd1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:58.368904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9269-2430-41f6-9839-5a90443c2873 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.369186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9269-2430-41f6-9839-5a90443c2873 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.369459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f9269-2430-41f6-9839-5a90443c2873 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.372103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 446/898] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:53:58.378792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.379721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.380222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.402368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:58.402974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42c3f7-f589-4360-a1ad-62c4e8323bd1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:58.404069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:58.431645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:58.441917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:58.443131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.443555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.444075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.455770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:53:58.465353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:58.466287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:53:58.467221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:58.469765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 453/899] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:53:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:53:58.476008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:58.490436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 447/900] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:53:58.510719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:58.595047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429791d-7226-4a4e-add0-8304a437a5ef None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:58.623182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c24f51-ca3f-4c6e-828b-46d93baa7d3c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:58.641154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:58.642080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.642450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.642785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.643939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:58.644310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:58.645756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:58.655827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:58.658366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 454/901] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:53:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:58.663095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:58.668154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:58.668154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.668692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.668692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.669475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:58.671065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:58.690355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 448/902] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:53:58 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:58.731478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:58.790103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe62e-5950-443b-8e70-cde667833060 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:58.835940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:58.837013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.837783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.838197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.851009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:58.852760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a328ca63-2c90-4387-b900-728aad51adf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:53:58.871554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:58.872686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:58.873879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:53:58.873879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:53:58.873879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:53:58.873879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:58.875867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 455/903] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:58.881645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:58.964200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7c1-61ba-4323-8444-4eacd0a51dd5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.964876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7c1-61ba-4323-8444-4eacd0a51dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.965410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7c1-61ba-4323-8444-4eacd0a51dd5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.967400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 456/904] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:53:58.973388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:58.973672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:58.973954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:58.992860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:53:58.994142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:53:58.996742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d683d4a-fc07-4db8-9a9e-008c616bc0b0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.040860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.041674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.042017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.042384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.043181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:59.043552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.045120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.057759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 449/905] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:59.063399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.071236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 457/906] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:53:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:53:59.094177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.122563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.126996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.182111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03ff30-57c9-4fa4-86dd-70ca4646f861 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.203561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c272f7-9168-4c5d-ba81-84ceb3e12972 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.219815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.221051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.221557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.222184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.230430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.232404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.239717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.240523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.240800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.241112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.241767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.243154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:53:59.250695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:59.251605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:53:59.252516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:53:59.255174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 450/907] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:53:59.256772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:53:59.257704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 12:53:59.258580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:53:59.262907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 458/908] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:53:59.264473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.264920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.362310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.372391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d0ce3-59a1-4917-b626-2a537709288c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.378568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509c4b-4717-4bcb-8de4-86f10856c6d3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.390333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.411053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.412058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.412494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.412905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.413887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:59.414442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.415830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.416255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.416682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.417054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.417431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.418239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:59.418765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.420694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.430439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 451/909] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:59.432757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.442235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 459/910] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:53:59.447536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.548989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863aaee1-e470-44e3-9205-c10d5791f920 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.579164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7b44d-1cb7-48c4-a52d-123f328fc71e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.585221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.586091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.586588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.587030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.588465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:53:59.588951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.590748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.601819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 452/911] 10.222.0.49 () {56 vars in 1114 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:53:59.603337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.619219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.619981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.620332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.621029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.647877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.649013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.711016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.713526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.715752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1682e-ba5f-4edb-9521-fcedfe33e382 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.752308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:53:59.753314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:53:59.753886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:53:59.754481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:53:59.755550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:53:59.757128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:53:59.786498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 453/912] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:53:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:53:59.829199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 460/913] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:53:59.834673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:53:59.943747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:53:59.959619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46d584-7679-49df-963f-9951a77f6be0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:53:59.999410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.000225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.000507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.000783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.011682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.013157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:00.032531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:00.033442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:00.034537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:00.034537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:00.034537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:00.034537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:00.034537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 454/914] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:00.039969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.091111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.142839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0076bc-6dc2-479e-8786-1d01c58711ac None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.177360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.178026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.178303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.178630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.179352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:00.179656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.181028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a6bfd-d7c7-4c87-9a96-d5dcf661d220 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.193078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 461/915] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:00.199257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.269756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890cd78-1df6-4e1d-8d43-b52b6dec6361 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.270381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890cd78-1df6-4e1d-8d43-b52b6dec6361 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.270860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890cd78-1df6-4e1d-8d43-b52b6dec6361 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.272735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 462/916] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:00.278237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.278787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.279335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.296887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:00.298463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:00.306928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1318225e-c8f3-4cb6-9cc4-481e53526be0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.345144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.345946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.346224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.346522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.355473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.356939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:00.371612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:00.372520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:00.373764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 455/917] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:00.376031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 463/918] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:00.378750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.381745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.404145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.446622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.490515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40d988-8c47-4f72-974f-aaf015916e14 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.510788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21270df8-3986-4edc-8ffb-ec5a2282013c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.526820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.527842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.528319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.528829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.529932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:00.530494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.532171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.546488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 456/919] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:00.558438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.558438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.558438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.558438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.558934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.559017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.559103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:00.567015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:00.568000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:00.569059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 464/920] 10.222.0.49 () {56 vars in 1132 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:00.575169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.631601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.672778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee1d6f-4a88-44f0-be12-49ddb4fe51c2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.709625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.710285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.710589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.710859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.711687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] RBAC: Adding query filter params (role.id=6f0e77cd25804da2a0081a565aa268bb, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=1bc929bde94d48e2b48e959d970ab1d1) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:00.711994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.713780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.718962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0366ef-b8ae-4326-b120-242c35fb2044 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.726362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 457/921] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/role_assignments?user.id=1bc929bde94d48e2b48e959d970ab1d1&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=6f0e77cd25804da2a0081a565aa268bb => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:00.757264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.761419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.762088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.762392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.762674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.763643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:00.764040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.765740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40daa5-da85-4913-a396-02f8a55046cf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.774483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 465/922] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:00.778146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:00.863168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2455f-b543-40e9-b979-c9da98080af1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.863512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2455f-b543-40e9-b979-c9da98080af1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.863736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2455f-b543-40e9-b979-c9da98080af1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.865413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 466/923] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:00.871200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.871441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.871701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.889504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:00.890779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:00.910345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786acc51-40ce-4521-af62-77d54c484aff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:00.946999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:00.973338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:00.973880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:00.974338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:00.974706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:00.975587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:00.975959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:00.977685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:00.978311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 467/924] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:00.984208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 458/925] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:00.990359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.001471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.111023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:01.119383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5da44e-5c03-4b5b-9911-8877710624b7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.139209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e95d0b-2905-4654-8407-59a915cb9f8b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.155423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.156186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.156527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.156863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.157695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.159164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.189887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 468/926] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:54:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:01.205561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.206901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.207419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.207867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.208761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.210459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:01.225683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:01.226847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:01.228298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:01.232953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 459/927] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:01.239294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.367728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1176daf-c132-4ae8-b937-385b76a97ca9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.404113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.404843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.405063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.405334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.406049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:01.406391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.407776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.419417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 469/928] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:01.425179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.495003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:01.558369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7765169-e5bf-4e09-adc8-340dd4af6fe6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.599258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.599967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.600248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.600522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.601243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:01.601552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.603042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.618907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 460/929] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:01.624669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.645289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:01.739902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e7e88-144a-4fa2-8a85-ac857e45f41b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.777453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.778108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.778408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.778682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.781022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] RBAC: Adding query filter params (interface=public, service_id=e4ff8c1e59354b2e888dc00a40bd100b, region_id=RegionOne) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:01.781340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.782859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e8c24-2414-40a1-a87b-8271553c6739 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.788646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 470/930] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:54:01 2022] GET /identity/v3/endpoints?service_id=e4ff8c1e59354b2e888dc00a40bd100b&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:01.794165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:01.897459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076371cf-7c32-476f-bb78-3f53b9c59afe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:01.911737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:01.935062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:01.935829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:01.936110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:01.936384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:01.937123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:01.938585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4cf1-d1c4-4859-ac8f-bd17c72846eb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:01.945263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 461/931] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:54:01 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:54:02.125310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03c37-1d4d-4b84-a647-eca6ad6c4cf9 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.125820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03c37-1d4d-4b84-a647-eca6ad6c4cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.126205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e03c37-1d4d-4b84-a647-eca6ad6c4cf9 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.127752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 471/932] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:02.133268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad020473-db92-4643-9322-9386b1eb889e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.133560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad020473-db92-4643-9322-9386b1eb889e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.133842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad020473-db92-4643-9322-9386b1eb889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.150880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ad020473-db92-4643-9322-9386b1eb889e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:02.152000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ad020473-db92-4643-9322-9386b1eb889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:02.219811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 462/933] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:02.243215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e03c37-1d4d-4b84-a647-eca6ad6c4cf9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:02.342576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:02.364562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e03c37-1d4d-4b84-a647-eca6ad6c4cf9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:02.400268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:02.401371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.401851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.402346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.403263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:02.405201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:02.407280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING keystone.server.flask.common [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 12:54:02.428680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.sql.core [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:54:02.428680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 12:54:02.428680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [parameters: {'id': '797c68802e8642fa89ccec4a2c796864', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 12:54:02.428680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80101) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 797c68802e8642fa89ccec4a2c796864.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 797c68802e8642fa89ccec4a2c796864. Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:54:02.446827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result.read() Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 12:54:02.447698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application yield resource Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise self.value Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application session.commit() Jun 30 12:54:02.448644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:54:02.449511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:54:02.450429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result.read() Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:54:02.451250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [parameters: {'id': '797c68802e8642fa89ccec4a2c796864', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:02.452174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:02.452944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 797c68802e8642fa89ccec4a2c796864. Jun 30 12:54:02.453838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:02.457469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 472/934] 10.222.0.49 () {60 vars in 1140 bytes} [Thu Jun 30 12:54:02 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:02.463579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad020473-db92-4643-9322-9386b1eb889e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:02.566618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad020473-db92-4643-9322-9386b1eb889e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:02.576003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:02.606049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:02.607307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.607787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.608266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.615664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:02.617382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e40f7d07-9c1d-4537-ab5f-09cbecc2fe68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:02.631590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:02.632461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:02.633326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:02.635785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 463/935] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:02.643698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:02.761215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e59b3-858a-4753-9052-d3670ecaf916 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:02.795785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:02.796447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.796725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.796995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.797743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:02.798042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:02.799704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b302185-d9d7-4f0b-8dfb-a8bafcde2974 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:02.803445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:02.821180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 473/936] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:02.973523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6c63-854d-41b3-a1a7-8c5e178fb055 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.974074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6c63-854d-41b3-a1a7-8c5e178fb055 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.974074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c6c63-854d-41b3-a1a7-8c5e178fb055 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:02.975609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 464/937] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:02.980966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:02.981255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:02.981535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.002628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:03.003823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:03.074875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 474/938] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:03.097973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c6c63-854d-41b3-a1a7-8c5e178fb055 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.207425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c6c63-854d-41b3-a1a7-8c5e178fb055 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.242736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.243560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.243832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.244095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.257744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7d57ee09-4c83-4180-81c8-8a614153c5af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:03.263115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:03.264349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:03.265237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 465/939] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:03.271460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.368089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661025d-5aeb-47a5-a158-a829953f3d42 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.368438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661025d-5aeb-47a5-a158-a829953f3d42 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.368810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661025d-5aeb-47a5-a158-a829953f3d42 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.371097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 466/940] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:03.376961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.377248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.377676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.381070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58322ab2-6b3f-4d90-b7c9-5b1a6d957a17 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.397235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:03.399099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:03.411883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.425067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.425792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.426525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.426814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.427721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:03.428142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.430023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.438683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.440702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 475/941] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:03.446437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.484416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 467/942] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:03.506945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.561986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26c296-67f4-442e-a7bd-cca3a5747bfe None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.597338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.598131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.598509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.598780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.610114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.610669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80f82b-7071-4cf1-8929-fd2306c1e610 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.611785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:03.637091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:03.639028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:03.640159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:03.640159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:03.640159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:03.640159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:03.640159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 476/943] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:03.644554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.647540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.648418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.648806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.649143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.649918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.651429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:03.664299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:03.666363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:03.668369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:03.675687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 468/944] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:03.675687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.688412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.699685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.752576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7469b-5fac-45a4-abe4-598a4c74aaca None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.780483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2619a-5437-4e7a-af2b-49632f1ab939 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.787998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.788854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.789301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.789714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.790595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:03.791058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.792567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.805367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 477/945] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:03.814247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.820068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.820793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.821077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.821352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.822207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:03.822391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.823778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.829188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 469/946] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:03.834493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:03.915049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.919061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170189e-948f-45e0-aab0-b42796c33d0f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.921498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:03.937235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab0b98-5898-422c-9770-1146128fc30a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:03.959190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.960116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.960398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.960672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.969987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.971640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.973812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:03.974592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:03.974937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:03.975273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:03.976053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:03.976471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:03.977831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-aeff011b-c75a-4b7a-aea0-b32415a3319f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:03.989636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:03.990741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:54:03.991710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:03.992258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 470/947] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:03.994719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 478/948] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:04.006552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.088083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122097e-9123-4d65-bce8-b522ef7a8dc2 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.088427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122097e-9123-4d65-bce8-b522ef7a8dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.088641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122097e-9123-4d65-bce8-b522ef7a8dc2 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.095049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 479/949] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:04.096614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.096913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.097182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.117055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:04.118269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:04.129437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba3009-f042-419c-8dab-a612c0f6aa25 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.165681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.166277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.166583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.166852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.167688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:04.167991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.169457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.190067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 471/950] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:54:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:04.197456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.208050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 480/951] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:04.209801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.240131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.308715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a569b-bd30-4f00-ab2c-5d9ffd32452b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.334757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.340630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b9c08-6bd8-45d5-a58a-94bfc17d5ae1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.345558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.346417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.346703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.346975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.358937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.360592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.377963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.378907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.379261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.379605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:04.380216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:04.381047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:04.382366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:04.382366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:04.382366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:04.382366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:04.382880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.382880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.383114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 472/952] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:04.387239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:04.399315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:04.401470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:04.403556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:04.410507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 481/953] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:04.410507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.450382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.494825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c4dd-0fdb-4be2-8953-ec0063bf66cf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.514699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa5bc3-def2-4a17-a1bd-e86e5e6dd0b0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.531741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.532504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.532784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.533159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.533884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:04.534187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.535605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.548711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 473/954] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:04.553854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.555023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.555692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.555967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.556235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.557155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:04.557462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.559751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.566030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 482/955] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:04.571547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.592676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.667640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82ac7a-ce78-48e9-b7dc-a19081a41c97 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.679233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52e5cc-27ba-490e-b029-7d744c2dcdff None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.688844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.704846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.705572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.705848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.706118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.714712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.716172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.720021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.720981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.721472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.721951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.722890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:04.723424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.724990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:04.735086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:04.736143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:04.737072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:04.737518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 474/956] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:04.738354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 483/957] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:04.738458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.739653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.827424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:04.847595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dca93f-9940-413a-8cd2-9920d65f9a37 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.849868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dad596-1d0d-43c3-a820-5a2a60aa383e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:04.885668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.886876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.887400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.887818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:04.887986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.889042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:04.889522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.889665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:04.890045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:04.890476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:04.891325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:04.891745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.892862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf36475-c4e8-480e-8b6f-305b5e16fe51 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:04.906078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 484/958] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:04.912197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:04.922367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 475/959] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:54:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:05.043606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018de96-0d86-4d3b-a5a0-a067adf055cb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:05.063004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0633a4-e44f-4aa9-8e1e-cf4cefebef11 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.063551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0633a4-e44f-4aa9-8e1e-cf4cefebef11 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.063551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0633a4-e44f-4aa9-8e1e-cf4cefebef11 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.069113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 476/960] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:05.070945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.071230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.071509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.093016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:05.093783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:05.094214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:05.095072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.095528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.095941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/1bc929bde94d48e2b48e959d970ab1d1/roles/6f0e77cd25804da2a0081a565aa268bb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.099479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=1bc929bde94d48e2b48e959d970ab1d1, role_id=6f0e77cd25804da2a0081a565aa268bb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:05.101095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:05.112506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:05.120627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 485/961] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:54:04 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/1bc929bde94d48e2b48e959d970ab1d1/roles/6f0e77cd25804da2a0081a565aa268bb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:54:05.137996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:05.197806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 477/962] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:05.220653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:05.345595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4035f-7731-4ddd-857a-f51417a9de3d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:05.381222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:05.382285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.382836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.383301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.384229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:05.385860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:05.400052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.sql.core [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:54:05.400052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 12:54:05.400052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [parameters: {'user_id': '1c5b801ce0804c00aa027ec2e02a9587', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 12:54:05.400052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80101) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result.read() Jun 30 12:54:05.408586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:54:05.409477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application yield resource Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application next(self.gen) Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise self.value Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application session.commit() Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 12:54:05.410500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.session.flush() Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application compat.raise_( Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 12:54:05.411550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 12:54:05.412735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application conn.query(q) Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application result.read() Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 12:54:05.413611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [parameters: {'user_id': '1c5b801ce0804c00aa027ec2e02a9587', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:05.414512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 12:54:05.415500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 12:54:05.416533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:05.422445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 486/963] 10.222.0.49 () {60 vars in 1131 bytes} [Thu Jun 30 12:54:05 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:05.424940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:05.535024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aeea2a-0d47-4785-8442-6476dfdd6ab3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:05.569257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:05.570304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.570773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.571184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.579099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:05.580793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-37d0425f-6923-44fb-a0b4-ab7d387688be None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:05.596804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:05.597990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 12:54:05.599188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:05.605886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 478/964] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:05.607787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:05.677266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:05.721574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aae7f2-5e4d-40ad-8870-c5806424e4a9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:05.756212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:05.757180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:05.757601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:05.758015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:05.758983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:05.759428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:05.761223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60681784-9c65-44d7-aac5-e1f074ab75a5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:05.804555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 487/965] 10.222.0.49 () {56 vars in 1101 bytes} [Thu Jun 30 12:54:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:06.075368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3f129-9f0e-4ef4-b79f-deff47fbef95 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:06.075928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3f129-9f0e-4ef4-b79f-deff47fbef95 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:06.076107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3f129-9f0e-4ef4-b79f-deff47fbef95 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:06.077668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 479/966] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:06.082763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0afa0-f13a-45b0-b3b0-1e05e81cb612 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:06.083058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0afa0-f13a-45b0-b3b0-1e05e81cb612 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:06.083335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0afa0-f13a-45b0-b3b0-1e05e81cb612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:06.100492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e1a0afa0-f13a-45b0-b3b0-1e05e81cb612 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:06.101998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e1a0afa0-f13a-45b0-b3b0-1e05e81cb612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:06.170775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 488/967] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:06.193203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3f129-9f0e-4ef4-b79f-deff47fbef95 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:06.221727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:06.309104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3f129-9f0e-4ef4-b79f-deff47fbef95 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:06.344295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:06.344877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:06.345158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:06.345429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:06.346170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:06.347581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63483b79-5b8c-4f9d-9154-932b5b4a5187 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:06.364026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 480/968] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:54:06 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:07.031728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bdd4-4f28-4931-a5bc-2a35f23a59d7 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.032303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bdd4-4f28-4931-a5bc-2a35f23a59d7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.032303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bdd4-4f28-4931-a5bc-2a35f23a59d7 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.033620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 489/969] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:07.039319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef814d-f936-43ae-953b-05e172ac6bca None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.039580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef814d-f936-43ae-953b-05e172ac6bca None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.039883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef814d-f936-43ae-953b-05e172ac6bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.057041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-adef814d-f936-43ae-953b-05e172ac6bca None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:07.058213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-adef814d-f936-43ae-953b-05e172ac6bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:07.134633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 481/970] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:07.157239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5bdd4-4f28-4931-a5bc-2a35f23a59d7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.208982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.220044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161464d-e0ce-464c-b98d-85b098b3f590 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.220332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161464d-e0ce-464c-b98d-85b098b3f590 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.220604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161464d-e0ce-464c-b98d-85b098b3f590 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.221961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 482/971] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:07.226910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.227191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.227461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.244644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:07.245826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:07.253215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.277435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5bdd4-4f28-4931-a5bc-2a35f23a59d7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.313482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.314558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.314836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.315108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.315818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.317210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.317408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 483/972] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:07.334892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:07.335895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:07.336769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 490/973] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:07.342742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.345599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.447582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecae560-9840-4008-af4c-02422a0bb592 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.448865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebde5-4aba-4d04-9bc9-b0b175238fdd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.464545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.484973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.485658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.485942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.486218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.486983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:07.487292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.488711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.491724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.492602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.492829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.493105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.499042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 484/974] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:07.499937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.508929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:07.509322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:07.517836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:07.519017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:07.520040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:07.521078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 491/975] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:07.526639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.613003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce20a56-1d92-4cba-a214-e73113bc6ba5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.632705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc97c6-4613-4365-9e66-2ffd3429e116 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.647083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.647846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.648130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.648407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.649116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:07.649494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.650964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.655860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 485/976] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:07.661589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.675172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.675832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.676107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.676381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.677140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:07.677443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.678841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.691855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 492/977] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:07.692768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.727289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.744299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.776967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46291-d606-4d5d-9c38-5df9878fc782 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.800373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fdddc-da1f-41a8-9a0a-9e78e58be432 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.812049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.812710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.812991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.813354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.814058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] RBAC: Adding query filter params (interface=admin, service_id=e4ff8c1e59354b2e888dc00a40bd100b, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:07.814548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.818359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.821725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 486/978] 10.222.0.49 () {56 vars in 1249 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/endpoints?service_id=e4ff8c1e59354b2e888dc00a40bd100b&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:07.827123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.842190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.843224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.843501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.844038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.855411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.856963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:07.875155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:07.876045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:07.876925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:07.876925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:07.876925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:07.876925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:07.876925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 493/979] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:07.881986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:07.929978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e40f63-438e-49e8-ae28-a686bf137f62 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:07.967154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:07.967839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:07.968123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:07.968398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:07.969094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:07.970517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:07.971762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:07.986356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 487/980] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:54:07.991907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.009809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66ba4d-38f3-4275-82fb-10ba18e28477 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.049230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.049902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.050183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.050487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.051220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:08.051523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.052909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.066862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 494/981] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:08.072833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.189428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d1713-ef53-40f5-b710-fdf6f93bda8f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.223302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.224118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.224395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.224665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.232500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.233955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3e25f7e6-8f00-4bfe-a437-c6ac80d40f0d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:08.248603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:08.249549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:08.250504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 488/982] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:08.259076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.361506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4176c-da0a-411e-8382-fa84e5b81581 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.395244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.395914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.396192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.396464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.397290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:08.397595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.399113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.406784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590578-5dba-44db-8e44-5494772961c5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.407048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590578-5dba-44db-8e44-5494772961c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.407323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f590578-5dba-44db-8e44-5494772961c5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.408699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 489/983] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:08.413714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.414001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.414283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.422352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 495/984] 10.222.0.49 () {56 vars in 1149 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:08.423397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.427058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.435527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:08.436986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:08.495945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.510852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 490/985] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:08.533200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.543827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367860af-9de3-414b-bdf9-62439cc83c48 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.582885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.583790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.584071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.584353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.640679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62211217-8f15-4b07-a190-08b8afbce5f2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.659655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.661132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.665016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.676148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.676956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.677305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.677563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.678165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.679586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:08.691321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:08.692332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:08.693161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 491/986] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:08.698859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.754816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.771090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 496/987] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:08.776506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.809257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24b7b6-3fe3-4d56-9dfe-d79f35600710 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.844361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.845018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.845298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.845569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.846274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:08.846595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.847976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.854370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 492/988] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:08.858663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.882895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129b471-9cd6-4844-bdb2-91f003596f9c None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.886210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:08.920125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:08.920912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:08.921192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:08.921467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:08.931627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:08.933055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:08.950572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:08.951601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:08.952533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:08.952533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:08.952533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:08.952533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:08.952533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 497/989] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:08.957766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:08.969119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a6ed7-6fb2-4684-8be1-4a5596aadfcd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:08.986486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:09.005624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.006584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.007042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.007520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.008437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:09.008782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.010151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.019938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 493/990] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:09.020970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:09.075155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461fac03-2a4a-4d3c-af9e-a9816ac17b03 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.111691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.112353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.112636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.112914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.113636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:09.113944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.115378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.127777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 498/991] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:09.132896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:09.134966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49c5bc-92d0-493a-a4a2-d318d2083e11 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.151615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:09.177962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.178651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.178932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.179203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.179987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] RBAC: Adding query filter params (interface=public, service_id=6162f0ef505d40dfa68e4c0be8c81689, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:09.180294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.181669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.186926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 494/992] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:54:09 2022] GET /identity/v3/endpoints?service_id=6162f0ef505d40dfa68e4c0be8c81689&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:09.192595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:09.239769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:09.240164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de3f38-e4fa-4382-bff2-e8bd0890bfb3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.279675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.280477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.280758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.281050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.288507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.289942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.302485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a85e-1174-4eb7-98ae-9a4d8d45fa20 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:09.304597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:09.305450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:09.306527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 499/993] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:09.311697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:09.342537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.343211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.343497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.343775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.344460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.346064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.352424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 495/994] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:54:09 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:54:09.391547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:09.418057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7445-0625-47a3-9d24-95dc90e428c7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.457403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.458102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.458512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.458786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.459578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:09.459883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.461340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf590d00-c4ba-424f-98b0-a7bbae3e7237 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.475168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 500/995] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:09.481229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:09.588226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8b037-0de4-4eb2-b795-aa273229d3fa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:09.597739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:09.624647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:09.625306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:09.625588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:09.625860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:09.626719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] RBAC: Adding query filter params (role.id=6f0e77cd25804da2a0081a565aa268bb, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=1bc929bde94d48e2b48e959d970ab1d1) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:09.627034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:09.628483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c810a2-72ef-4aa8-8a42-9694acbf0a95 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:09.634865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 496/996] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:09 2022] GET /identity/v3/role_assignments?user.id=1bc929bde94d48e2b48e959d970ab1d1&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=6f0e77cd25804da2a0081a565aa268bb => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:10.009126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f568ce9-7af8-4415-a0fa-4cc235d4fef6 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.009539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f568ce9-7af8-4415-a0fa-4cc235d4fef6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.009717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f568ce9-7af8-4415-a0fa-4cc235d4fef6 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.011153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 501/997] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:10.017145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.017402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.017708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.035628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:10.036880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:10.108417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 497/998] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:10.131749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f568ce9-7af8-4415-a0fa-4cc235d4fef6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:10.234634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f568ce9-7af8-4415-a0fa-4cc235d4fef6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:10.272931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:10.273745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.274022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.274291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.274985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:10.276344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:10.279433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:10.290198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:10.291408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 12:54:10.292427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:10.295336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 502/999] 10.222.0.49 () {56 vars in 1126 bytes} [Thu Jun 30 12:54:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:10.301321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:10.405076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a316ee-4c42-44a6-be44-8b6b7fd9828e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:10.406641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:10.445211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:10.445884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.446164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.446472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.447186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:10.447495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:10.448883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:10.453900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 498/1000] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:10.460011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:10.565072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f61ce-95d6-4631-bd4b-9d255e33e729 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:10.598564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:10.599212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.599489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.599753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.600444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:10.600740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:10.602103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbde5dd-c4d1-479f-a421-a1cac7aba30d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:10.607338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 503/1001] 10.222.0.49 () {56 vars in 1122 bytes} [Thu Jun 30 12:54:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:10.613027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:10.716580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e691125-99ca-4997-bc41-899e9b122eb1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:10.751903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:10.752991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:10.753477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:10.753979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:10.755017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:10.756720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccb1f12-7cfb-4ee0-8eb7-99880795acfd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:10.774571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 499/1002] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:54:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:11.404293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4263-d2ae-4231-92c2-1f54a698469d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.404830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4263-d2ae-4231-92c2-1f54a698469d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.404830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4263-d2ae-4231-92c2-1f54a698469d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.406440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 504/1003] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:11.411611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934929b-8e76-4ad4-856a-80155bcc7f18 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.412028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934929b-8e76-4ad4-856a-80155bcc7f18 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.412205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934929b-8e76-4ad4-856a-80155bcc7f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.429407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8934929b-8e76-4ad4-856a-80155bcc7f18 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:11.430593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8934929b-8e76-4ad4-856a-80155bcc7f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:11.452542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:11.518447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 500/1004] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:11.537688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d4263-d2ae-4231-92c2-1f54a698469d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:11.604245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:11.644556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d4263-d2ae-4231-92c2-1f54a698469d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:11.661600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608864-9791-4fec-997d-88a21a97f009 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.661865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608864-9791-4fec-997d-88a21a97f009 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.662152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7608864-9791-4fec-997d-88a21a97f009 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.663664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 501/1005] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:11.668302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.668602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.668872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.678087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:11.678926 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.679192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.679467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.680117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:11.681785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:11.691354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:11.692555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:11.694035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:11.695005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:11.695813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 505/1006] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:11.701146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:11.762975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 502/1007] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:11.785034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:11.804614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113dd311-8934-4a9b-a5f6-03197e93d447 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:11.831189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:11.843637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:11.844298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.844573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.844848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.845549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:11.845853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:11.847245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:11.852222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 506/1008] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:11.857913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:11.869331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:11.890097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae94714-7fa1-4eca-8166-705dee4e395b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:11.926277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:11.927144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:11.927385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:11.927655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:11.928705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:11.929683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:11.941734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:11.943111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:11.943942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 503/1009] 10.222.0.49 () {56 vars in 1105 bytes} [Thu Jun 30 12:54:11 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:11.948703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:11.970195 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826cc61-a708-460e-a90a-0102a031a396 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:12.004939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:12.005602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.005882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.006149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.006944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:12.007253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:12.008612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:12.014210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 507/1010] 10.222.0.49 () {56 vars in 1112 bytes} [Thu Jun 30 12:54:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:12.019934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:12.061428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d13145-5432-4f5d-affd-83cb75c535b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:12.076152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:12.097953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:12.098641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.098918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.099187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.099890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:12.100191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:12.101561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:12.111030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 504/1011] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:12.111998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:12.124699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:12.129108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f186799-32de-4b9d-9f61-c6f64790fae5 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:12.164595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:12.165246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.165525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.165796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.166563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:12.167922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1852a3f9-d821-4232-b811-adc623cb7266 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:12.190746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 508/1012] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:54:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:54:12.233073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2261225-cf7d-4597-8af0-ddde8ba8960d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:12.280769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:12.281458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.281741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.282014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.282772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:12.283088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:12.284493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb989fb-9a08-45e3-9fd8-06519d4c38f0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:12.298459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 505/1013] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:12.845368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0235-0e17-442c-9001-aa69204c5ad0 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.845892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0235-0e17-442c-9001-aa69204c5ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.846006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e0235-0e17-442c-9001-aa69204c5ad0 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.847434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 509/1014] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:12.852778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:12.853171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:12.853339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:12.871508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:12.872852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:12.938348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:12.950858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 506/1015] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:12 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:12.971545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e0235-0e17-442c-9001-aa69204c5ad0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:13.077082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e0235-0e17-442c-9001-aa69204c5ad0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:13.090610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:13.113795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:13.114641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:13.114922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:13.115194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:13.116371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:13.117808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a45ff-1eef-4b11-8a38-5f9ccb76e8a5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:13.119617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 510/1016] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:13.125444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:13.228073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919883d-e97f-4b86-bc36-a53383c39e29 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:13.272254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:13.273164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:13.273618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:13.274059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:13.275066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:13.276742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e269e5-d7a8-4e25-bada-611c0dfb7482 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:13.305814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 507/1017] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:54:13 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:54:14.327191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b8d2-0c6c-4768-85db-0e476fcec155 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.327764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b8d2-0c6c-4768-85db-0e476fcec155 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.327764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1b8d2-0c6c-4768-85db-0e476fcec155 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.330361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 511/1018] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:14.330806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29943831-52a1-4c94-9909-2421503ca81d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.331195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29943831-52a1-4c94-9909-2421503ca81d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.331368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29943831-52a1-4c94-9909-2421503ca81d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.333010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 508/1019] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:14.334200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.334584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.334923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.338309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.338622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.338904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.352778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:14.353971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:14.355896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:14.357115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:14.407015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:14.428014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 512/1020] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:14.432710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 509/1021] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:14.452053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:14.455821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:14.563098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71252deb-9c9d-48b2-9f90-2946a1fbc785 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:14.564328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37b750-6fa8-4dd6-be41-0299588e278d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:14.573428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:14.597183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:14.598001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.598280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.598773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.603211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:14.603876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.604155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.604425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.605140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:14.606551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:14.609493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:14.617700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:14.618854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:14.619876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:14.623846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 513/1022] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:54:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:14.624460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 510/1023] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:54:14 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:54:14.624739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:14.743681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f07e37-07ec-467f-9a25-44959f3c1676 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:14.787122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:14.787802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.788063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.788330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.789090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:14.789389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:14.790835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:14.799618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 514/1024] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:14.805411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:14.852545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:14.913179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3bba6-2437-40a5-8ec4-5ec54e7a19f9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:14.947939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:14.948736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:14.949010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:14.949277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:14.963443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:14.964958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:14.982954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:14.983919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:14.984782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:14.984782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:14.984782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:14.984782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:14.984782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 511/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:14.991438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.022380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.093849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef413a-f927-4432-816a-cb38434b97dd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.129447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.130111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.130412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.130683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.131398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:15.131701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.133068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.146841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 515/1026] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:15.152265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.247734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.257681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af0915-ddde-4233-9048-c920971e67a3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.294142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.295008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.295286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.295553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.303818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.305259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-76c35346-f1a3-468e-a5b1-d12e485670d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:15.319179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:15.320118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:15.321098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:15.326400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 512/1027] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:54:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:15.326554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.373949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4b95-e992-45fe-b18f-c8988fde21fc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.374583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4b95-e992-45fe-b18f-c8988fde21fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.374838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea4b95-e992-45fe-b18f-c8988fde21fc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.376498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 513/1028] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:15.381444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.381656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.381924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.399581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:15.400866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:15.424710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.438286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329fa67-72a5-4302-b84d-c5ae3737b696 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.473818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.474516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.474800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.478497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 514/1029] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:15.478597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.478597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:15.478597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.478597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.499062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 516/1030] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:54:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:15.499616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.504266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.596419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.608571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3c0ce-98f3-437e-9975-ef6eff9152d9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.615248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c91bb-6b89-494b-93c2-be4ca230a0fc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.645140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.646028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.646501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.646784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.651534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.652333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.652614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.652893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.654088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.656002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.658138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 515/1031] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:15.663628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.666503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.724963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.726459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.782201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ad2e6-66b6-4744-a3e6-cb897f5267bb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.820691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.821372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.821655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.821931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:15.822687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:15.824171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:15.839855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 517/1032] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:15 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:54:15.845734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:15.849193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 516/1033] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:54:15 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:54:15.898109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:15.960461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5991a790-08a6-4104-980c-d1d63405f2f0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:15.997961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:15.998793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:15.999078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:15.999351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.010946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.012400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:16.030861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:16.031965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:16.033047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:16.033047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:16.033047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:16.033047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:16.033047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 518/1034] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:16.038717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:16.051941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:16.149748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982e027-79ee-4e02-9c4f-ae5e66aa8c6b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:16.187313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:16.187992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.188273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.188554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.189289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:16.189592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.191018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.208692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 517/1035] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:16.209666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:16.322506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:16.327019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992f1d6-6b70-48d4-8cc0-5f9870e14faf None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:16.368292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:16.369131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.369430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.369704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.377224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.378703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:16.394411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:16.395630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:16.396522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 519/1036] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:16.399671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:16.473443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:16.508123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba3783-835b-4bc8-991d-67d90070bea6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:16.544990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:16.545656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.545939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.546213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.547024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:16.547335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.548783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67987a2-bc7e-44b6-aee6-50d019fae953 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.561177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 518/1037] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:16.567244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:16.668212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f901ad-b68e-492e-9ccd-f162f294eb69 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.668566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f901ad-b68e-492e-9ccd-f162f294eb69 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.668725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f901ad-b68e-492e-9ccd-f162f294eb69 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.670285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 519/1038] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:16.673390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf615-b557-44b5-acd6-df1109d38b8e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:16.675196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.675457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.675753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.692456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:16.693641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:16.709260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:16.709910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.710200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.710506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.711328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] RBAC: Adding query filter params (role.id=6f0e77cd25804da2a0081a565aa268bb, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:16.711642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.713420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.720153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 520/1039] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=6f0e77cd25804da2a0081a565aa268bb => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:16.766914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 520/1040] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:16.789732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:16.817685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:16.901504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85965a-a435-44b5-b2fd-d24a0035cf64 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:16.935936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:16.936737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:16.937018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:16.937293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:16.937944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:16.939338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:16.951517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:16.952497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:16.953325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 521/1041] 10.222.0.49 () {56 vars in 1105 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:16.959699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:17.024570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:17.069391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba8a7f6-d800-48e3-8253-08dd584b03f2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:17.103237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:17.103907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.104182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.104453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.105168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:17.105472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:17.106935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:17.112121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 521/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:17.117740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:17.204497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:17.220083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c7e6b-ba89-4148-8919-902423dfb1d2 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:17.253288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:17.254029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.257741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.257741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.257741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:17.257741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:17.257741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:17.264325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 522/1043] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:17.271019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:17.375300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d548c-d156-44fe-be9b-1d8c096c9805 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:17.399171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:17.412391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:17.413090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.413312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.413586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.414390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] RBAC: Adding query filter params (interface=public, service_id=496cd5c9c91e4349b91e3a0a64f1cd27, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:17.414717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:17.416084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b7f3c-416a-427f-992e-226e535d0e40 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:17.421608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 522/1044] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:54:17 2022] GET /identity/v3/endpoints?service_id=496cd5c9c91e4349b91e3a0a64f1cd27&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:17.427012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:17.528914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a427a9e-c14d-4b67-99c7-332e7c7771f0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:17.562516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:17.563147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.563423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.563689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.564364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:17.565744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21de9a7-50c4-481d-b301-c50cf830ea09 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:17.572821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 523/1045] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:54:17 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 12:54:17.951166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e248d-dbfb-4888-ba1c-d0031a4530cf None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.951569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e248d-dbfb-4888-ba1c-d0031a4530cf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.951670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e248d-dbfb-4888-ba1c-d0031a4530cf None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.953227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 523/1046] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:17.958888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:17.959284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:17.959781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:17.978218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:17.979433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:17.989748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:18.054376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 524/1047] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:18.076889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e248d-dbfb-4888-ba1c-d0031a4530cf None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.179712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e248d-dbfb-4888-ba1c-d0031a4530cf None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:18.213861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:18.214587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.214865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.215132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.225653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:18.233247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:18.234190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:18.240448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 524/1048] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:18.241152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.350134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b946a-fdff-4d44-86e9-b335f2efd07d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:18.383744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:18.384409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.384685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.384954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.385692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:18.385991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:18.387386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:18.395770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 525/1049] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:18.401364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.447911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:18.502648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f8bd5-7be4-441c-afaf-e13ab5a28962 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:18.535665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:18.536533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.536811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.537079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] PATH_INFO: `/v3/users/95b9b6e1c1a343629a2c75202c0444b2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.550614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] RBAC: Authorizing `identity:get_user(user_id=95b9b6e1c1a343629a2c75202c0444b2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:18.552070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81af99-b032-4eba-b00f-a5cca0aba684 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:18.553928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 525/1050] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/users/95b9b6e1c1a343629a2c75202c0444b2 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:18.559571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.569055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:18.662680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d2df6-b956-4f5a-8f83-66facb53e1f6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:18.698006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:18.698676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.698949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.699220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.720850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec8b6-6e32-4309-98b3-610d2bdd225e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.721076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec8b6-6e32-4309-98b3-610d2bdd225e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.721354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaec8b6-6e32-4309-98b3-610d2bdd225e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.722881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 526/1051] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:18.727595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.727928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.728135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.745665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:18.746897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:18.773679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:18.775090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:18.784931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:18.818430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 527/1052] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:18.841433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.875194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:18.893157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 526/1053] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:18.894057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:18.943506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca43c7-409d-422e-9cfb-e1714f4bbe61 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:18.980338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:18.981163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:18.981448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:18.981722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:18.993332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:19.001486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6bec9-d036-4df4-913a-5c5eb8718364 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:19.002221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:19.003338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:19.004211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 528/1054] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:19.009528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.012043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.044763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.045908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.046386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.046830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.048140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.049585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.051489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 527/1055] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:18 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:19.057226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.109789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.118291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8bf9d-7840-4c4b-b62e-125354b45524 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.154015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.154708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.154992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.155245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.156017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:19.156326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.157694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.164672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f70f6-2559-45bc-90ad-48ca29bd5310 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.166208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 529/1056] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:19.172355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.201760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.202436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.202719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.202992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.203816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=3d532c007e2b42f68d221e03c815ac04, user.id=95b9b6e1c1a343629a2c75202c0444b2) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:19.204127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.205550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.218451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 528/1057] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/role_assignments?user.id=95b9b6e1c1a343629a2c75202c0444b2&scope.project.id=3d532c007e2b42f68d221e03c815ac04&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:19.291514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758be74-5f68-48f0-be37-cb1c06a530c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.330457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.331168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.331447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.331703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.344675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.346163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:19.365063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:19.366056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:19.367396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:19.367396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:19.367396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:19.367396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:19.368637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 530/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:19.372416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.415908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.481438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f95c8-3b84-46d0-a95d-5c6f03b82da7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.516090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.517033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.517453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.517864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.518750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:19.519211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.520708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7a1f7-100e-4f5f-87a9-6de182cf9e4d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.533481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 529/1059] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:19.538884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.551738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.580166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0963-d1c1-4f06-82ba-0c23a9eeb220 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.580537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0963-d1c1-4f06-82ba-0c23a9eeb220 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.580720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b0963-d1c1-4f06-82ba-0c23a9eeb220 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.586277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 530/1060] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:19.587289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.587966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.588271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.606608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:19.607939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:19.649286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42cc1d-c7e4-47c4-bcc0-0fb7baddce10 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.684629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.685477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.685751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.686019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.686699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 531/1061] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:19.694355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.695913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.706923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:19.710298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:19.711242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:19.712168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 531/1062] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:19.717735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.741862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.792224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:19.828431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32915848-4dba-4228-84ef-848dca84dc51 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.832738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77efca7-b6c4-485f-ac7c-57fb7f7322ca None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:19.865331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.866009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.866291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.866623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.867430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:19.867737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.868840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:19.869177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.869634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:19.869916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:19.870189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:19.870887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:19.872368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:19.884766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:19.885939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 12:54:19.886867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:19.893184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 532/1063] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:19.893343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 532/1064] 10.222.0.49 () {56 vars in 1105 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:19.893423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.894197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:19.994648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.006904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da4763-f008-4ed0-9089-188b037817a6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.019389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264a9a6-c50f-4914-a8a6-db9da3b2e3b9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.040041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.040827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.041102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.041359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.047515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.053384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.054866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.057425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.058081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.058389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.058666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.059372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:20.059678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.061088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.071111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 533/1065] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:20.071922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:20.072278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:20.073113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:20.074625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:20.074625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:20.074625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:20.074625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:20.074625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 533/1066] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:20.079297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.178637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2b36-114a-4ce5-b42d-680993b74be7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.182618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e16bd8-01bd-44f5-b2f3-09bcdcbe7b05 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.214607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.215274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.215552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.215816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.216550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] RBAC: Adding query filter params (type=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:20.216852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.217089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.217744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.218022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.218226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.218342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.219022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:20.219409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.220738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.228350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 534/1067] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:20.229335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.233122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 534/1068] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:20.238750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.243345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.289512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.339271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70318b5b-5721-40df-9533-41f0dabda55d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.351905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3f714-0b55-4cb5-a96f-3451d32392cd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.373778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.374472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.374814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.375087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.375839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.377256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.390098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.390992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.391272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.391544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.398036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 535/1069] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:54:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:20.406614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.406614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:20.415470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:20.416456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:20.417548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 535/1070] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:20.422913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.536944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e48eed-05d9-4322-8f4b-3cfb4d204369 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.544936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.576889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.577565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.577850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.578125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.579037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:20.579402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.580855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6558314-307c-4af5-bf69-af580261d69d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.594413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 536/1071] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:20.600489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:20.664131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:20.703910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d92f78-6d8b-4003-956f-e6a5dac73dfb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:20.738636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:20.739527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:20.739803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:20.740071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/6f0e77cd25804da2a0081a565aa268bb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:20.742283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=2a0b0da6c54341ff8fb66f48a0a32b70, role_id=6f0e77cd25804da2a0081a565aa268bb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:20.743744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750dd43-46bb-41b4-af65-fcd8d404ef32 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:20.761650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 536/1072] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:54:20 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/2a0b0da6c54341ff8fb66f48a0a32b70/roles/6f0e77cd25804da2a0081a565aa268bb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:54:21.225812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063089-aa4c-44de-9e84-28cb06624ed8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.226231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063089-aa4c-44de-9e84-28cb06624ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.226355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063089-aa4c-44de-9e84-28cb06624ed8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.227947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 537/1073] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:21.233194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.233447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.233750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.250647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:21.251819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:21.322885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 537/1074] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:21.345516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063089-aa4c-44de-9e84-28cb06624ed8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:21.453813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063089-aa4c-44de-9e84-28cb06624ed8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:21.488276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:21.488964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.489256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.489524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.499980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:21.507573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:21.508509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:21.515207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 538/1075] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:21.517271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:21.579256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:21.629394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70ec57-652f-4f3b-803a-317069624dd0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:21.665743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:21.666694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.667189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.667657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.668582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:21.669081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:21.671186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:21.679999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 538/1076] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:21.686406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:21.715342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:21.792533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba3c89-3f7c-4073-8f54-31bf1bf7ece9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:21.825482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:21.826432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.826710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.826977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] PATH_INFO: `/v3/users/95b9b6e1c1a343629a2c75202c0444b2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.828151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] RBAC: Authorizing `identity:get_user(user_id=95b9b6e1c1a343629a2c75202c0444b2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:21.829591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:21.831396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 539/1077] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:21 2022] GET /identity/v3/users/95b9b6e1c1a343629a2c75202c0444b2 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:21.837288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:21.949145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eeaf4c-e1d1-4f57-8c7e-ff4b60963279 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:21.967002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:21.987069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:21.987946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:21.988222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:21.988491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:21.989777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:21.991257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5b88a-0968-44b9-924c-1edbab11bd0f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:21.993182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 539/1078] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:21 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:21.999184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:22.096156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:22.104806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5895254-99e9-41f3-ba37-a594db6388e8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:22.140992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:22.141881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.142158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.142453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04/users/95b9b6e1c1a343629a2c75202c0444b2/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.144365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] RBAC: Authorizing `identity:create_grant(project_id=3d532c007e2b42f68d221e03c815ac04, user_id=95b9b6e1c1a343629a2c75202c0444b2, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:22.145797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f147a9-5478-4aea-b2d2-57033939f62c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:22.164102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 540/1079] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:54:21 2022] PUT /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04/users/95b9b6e1c1a343629a2c75202c0444b2/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:54:22.492105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8848-529e-4739-bfe0-7bb8efd101ca None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.492390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8848-529e-4739-bfe0-7bb8efd101ca None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.492662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8848-529e-4739-bfe0-7bb8efd101ca None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.494066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 540/1080] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:22.499669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952ec3-0061-4ef8-b6a3-3fa27aa68aa8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.499959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952ec3-0061-4ef8-b6a3-3fa27aa68aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.500204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a952ec3-0061-4ef8-b6a3-3fa27aa68aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.519045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3a952ec3-0061-4ef8-b6a3-3fa27aa68aa8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:22.520254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a952ec3-0061-4ef8-b6a3-3fa27aa68aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:22.591664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 541/1081] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:22.614143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8848-529e-4739-bfe0-7bb8efd101ca None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:22.717385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8848-529e-4739-bfe0-7bb8efd101ca None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:22.750907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:22.751471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.751746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.752015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.767911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceed8fe-59ad-4d5d-826e-48a2a6de0421 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.768451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceed8fe-59ad-4d5d-826e-48a2a6de0421 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.768451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceed8fe-59ad-4d5d-826e-48a2a6de0421 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.770054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 542/1082] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:22.774612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:22.774902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:22.775178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:22.792557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:22.793741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:22.809845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:22.829644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:22.831135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:22.870374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 543/1083] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:22.890555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:22.906196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:22.936722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 541/1084] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:22.942484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.006179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ca04b-a526-4eee-9f5a-b5e0ecbf4aef None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.037469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:23.047270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.048225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.048738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.049013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.055446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ad407-9539-4149-bf93-77721659cc5a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.061461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:23.069635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:23.070624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:23.072354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 544/1085] 10.222.0.49 () {56 vars in 1120 bytes} [Thu Jun 30 12:54:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:23.076799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.093886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.094739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.095022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.095295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.106671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.108582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:23.127327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:23.128283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:23.129182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:23.129182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:23.129182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:23.129182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:23.130713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 542/1086] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:23.134312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.153427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:23.179239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8e75f3-d22e-49e3-a411-fac33575db1e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.217331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.218007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.218283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.218583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.219339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:23.219641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.221023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.235111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 545/1087] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:23.236522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.239945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e201fd0-72cc-4120-bf8e-2af9790745a6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.277418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.278082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.278387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.278745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.279657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:23.279973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.281327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.284112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:23.302356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 543/1088] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:23.355512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6338f3e-fc9a-4598-ac3e-0226d79d9895 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.393128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.393946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.394259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.394630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.397781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:23.407079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.408570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:23.427251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:23.428232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:23.434000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:23.434000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:23.434000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:23.434000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:23.434000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 546/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:23.436333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.543064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eedc7d-0bcc-4840-b8c4-a66ae2ea995d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.576729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.577389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.577665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.577936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.578676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:23.578984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.580347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.593320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 544/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:23.598848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.699233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75323d67-1451-4334-bfc6-af5e9f54c200 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.734829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.735679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.735955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.736228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.744006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.745459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:54:23.759526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:23.760652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:23.762072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 547/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:23.766669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.825778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:23.870735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeb655-7787-43c6-9bf1-56ef11d25f04 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:23.906438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:23.907098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:23.907372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:23.907637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:23.908456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:23.908756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:23.910171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0ad84-9efb-4654-909b-8219f76226a2 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:23.927882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 545/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:23.933880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:23.986468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.036791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72bb9-f44a-4c91-b7b8-c632d40f1ead None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.071546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.072210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.072519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.072795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.145149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.146660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.193613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.217585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2e857-c67b-4525-9f83-9a160ec0ed1c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.218100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2e857-c67b-4525-9f83-9a160ec0ed1c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.218100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2e857-c67b-4525-9f83-9a160ec0ed1c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.219584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 546/1093] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:54:24.224602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.224857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.225145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.241937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:24.243156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:24.253656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 548/1094] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:24.255495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.314384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 547/1095] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:54:24.336560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.356654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac641b3-2330-448c-a22e-c8c3f80bca18 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.384183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.391304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.392086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.392365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.392636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.403917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.405314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.413617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:24.423606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:24.424516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:24.425400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:54:24.425400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:24.425400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:54:24.425400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:24.426478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 549/1096] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:24.430633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.444435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa82e6-19b1-45f5-a192-966cad17cd1f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.477704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.478535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.478814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.479085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.489096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:24.496712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:24.497803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:24.498821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 548/1097] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:24.504193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.530039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a340203-2189-4402-9627-31e0ca4dfcfe None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.563886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.564534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.564810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.565081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.565820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:24.566122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.567526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.578534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 550/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:24.584293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.604825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effa8a2-5642-45da-838b-6e208a7f2c11 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.607495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.639843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.640505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.640792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.641061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.641654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.641833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:24.642106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.643536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.652082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 549/1099] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:24.658512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.681383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fbad4-71c0-4929-95f6-2fb3180dc1a1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.714506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.715318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.715594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.715860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.723033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.724479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:54:24.738577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:24.739536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:24.740407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 551/1100] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:24.745468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.760066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a843c-5f81-4ffb-8bd4-b38d9b14ad99 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.793843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.794918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.794918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.795290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] PATH_INFO: `/v3/users/95b9b6e1c1a343629a2c75202c0444b2` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.796459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] RBAC: Authorizing `identity:get_user(user_id=95b9b6e1c1a343629a2c75202c0444b2)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.797878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.799547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 550/1101] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/users/95b9b6e1c1a343629a2c75202c0444b2 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:24.805213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.840476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.847499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800896-8cca-4b86-8bac-ed4fca5ea307 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.867406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:24.882498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.883154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.883428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.883739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:24.884517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:24.884822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:24.886240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:24.897683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 552/1102] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:24.903745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:24.905401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a5e1d-d030-453a-87d8-41509992089a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:24.942050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:24.942657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:24.942940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:24.943240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:25.003890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97a8cf-c690-4c0e-b11c-f7afc2379b6a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:25.010763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:25.012212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:25.037689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:25.038538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:25.038651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:25.038964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:25.039747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] RBAC: Adding query filter params (role.id=6f0e77cd25804da2a0081a565aa268bb, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=2a0b0da6c54341ff8fb66f48a0a32b70) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:25.040054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:25.041479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:25.043882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:25.047669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 553/1103] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/role_assignments?user.id=2a0b0da6c54341ff8fb66f48a0a32b70&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=6f0e77cd25804da2a0081a565aa268bb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:25.116823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 551/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:54:25.123232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:25.173774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:25.235542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09aa5d-6fe7-45a7-b69f-aafd312295e6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:25.271842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:25.272720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:25.273002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:25.273275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:25.274527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:25.275982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c1f14-9541-427a-9bf2-881a67a7e14a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:25.277910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 554/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:25 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:25.284022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:25.384269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16b837-577d-492d-a6f0-971c1904a93b None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:25.421074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:25.421732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:25.422012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:25.422289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:25.423155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=3d532c007e2b42f68d221e03c815ac04, user.id=95b9b6e1c1a343629a2c75202c0444b2) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:25.423471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:25.424909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6908ec8-8370-48c9-920b-3bbdb3468f42 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:25.429594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:25.436752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 552/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:25 2022] GET /identity/v3/role_assignments?user.id=95b9b6e1c1a343629a2c75202c0444b2&scope.project.id=3d532c007e2b42f68d221e03c815ac04&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:27.494061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278adaa-5c76-4781-a524-a77e98b4c4b7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:27.494652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278adaa-5c76-4781-a524-a77e98b4c4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:27.494652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278adaa-5c76-4781-a524-a77e98b4c4b7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:27.496008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 555/1107] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:27.502501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:27.503206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:27.503383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:27.531088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:27.532809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:27.612262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 553/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:27.637712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278adaa-5c76-4781-a524-a77e98b4c4b7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:27.766791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278adaa-5c76-4781-a524-a77e98b4c4b7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:27.780080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:27.817708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:27.818478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:27.818755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:27.819007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:27.820309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:27.821773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c26f1-d20f-4564-8952-8469262b2fc6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:27.823487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 556/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:27.829704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:27.931465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b1c1-0937-4c1d-afa1-ae9e858b44c9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:27.959957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:27.970885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:27.971693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:27.972010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:27.972280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:27.973360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:27.975175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14b986-1bbc-46a1-994b-f06ccef8631a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:27.997057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 554/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:54:27 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:30.052814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a86b-8d7a-450e-9d8d-046fc7316a25 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:30.055675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a86b-8d7a-450e-9d8d-046fc7316a25 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:30.055675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a86b-8d7a-450e-9d8d-046fc7316a25 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:30.055675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 557/1111] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:30.061264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:30.062000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:30.062133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:30.090914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:30.092374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:30.164299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 555/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:30.196930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a86b-8d7a-450e-9d8d-046fc7316a25 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:30.307272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a86b-8d7a-450e-9d8d-046fc7316a25 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:30.342284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:30.343173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:30.343452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:30.343725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] PATH_INFO: `/v3/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:30.345250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] RBAC: Authorizing `identity:get_role(role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:30.346663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:30.348031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 558/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:30 2022] GET /identity/v3/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:30.354681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:30.459481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccef45c-9eb4-498c-a092-f987e3a802cd None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:30.474906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:30.498835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:30.499702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:30.499979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:30.500250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] PATH_INFO: `/v3/users/e2e931531b734295bf0e94b1aeb8566f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:30.513429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] RBAC: Authorizing `identity:get_user(user_id=e2e931531b734295bf0e94b1aeb8566f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:30.514924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:30.516542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 556/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:30 2022] GET /identity/v3/users/e2e931531b734295bf0e94b1aeb8566f => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:30.523535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:30.583502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:30.644571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dae11-e1cf-413d-a752-e3ed3820d971 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:30.682303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:30.682959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:30.683329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:30.683599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:30.751595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:30.753040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:30.804258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:30.859940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 559/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:30.869648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:30.974653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5c1fd-7541-4dfa-b1fa-795bd817168d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:31.012349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:31.013248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:31.013613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:31.013889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:31.015093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:31.016525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0dc9b-e4cb-4aac-996e-55bc5a932dd1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:31.018249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 557/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:30 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:31.024793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:31.140643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77603fa5-f593-4588-b902-b895e344d861 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:31.179578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:31.180500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:31.180755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:31.181107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:31.182197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=3d532c007e2b42f68d221e03c815ac04, user.id=e2e931531b734295bf0e94b1aeb8566f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:31.182698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:31.185066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828db066-c477-4e10-bbfa-bf98cb3b5280 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:31.194366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 560/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:31 2022] GET /identity/v3/role_assignments?user.id=e2e931531b734295bf0e94b1aeb8566f&scope.project.id=3d532c007e2b42f68d221e03c815ac04&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:33.148296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d71b-29bf-4880-bdb1-c90dd61ea604 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.148888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d71b-29bf-4880-bdb1-c90dd61ea604 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.148888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7d71b-29bf-4880-bdb1-c90dd61ea604 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.150210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 558/1118] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:33.156640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.157399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.157535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.184420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:33.185985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:33.193377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:33.263217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 561/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:33.286013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d71b-29bf-4880-bdb1-c90dd61ea604 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:33.331118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:33.392349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7d71b-29bf-4880-bdb1-c90dd61ea604 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:33.427834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:33.428609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.428884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.429149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] PATH_INFO: `/v3/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.430744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] RBAC: Authorizing `identity:get_role(role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:33.432105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:33.433478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 559/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:33 2022] GET /identity/v3/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:33.440378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:33.551549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce1dbc-dc9b-48e1-8e14-d151d8e95137 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:33.586619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:33.587798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.588114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.588388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] PATH_INFO: `/v3/users/e2e931531b734295bf0e94b1aeb8566f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.589706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e931531b734295bf0e94b1aeb8566f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:33.591174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:33.592821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 562/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:33 2022] GET /identity/v3/users/e2e931531b734295bf0e94b1aeb8566f => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:33.601984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:33.710939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc50dd8-ce54-4569-b1ab-05265d01b606 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:33.720325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:33.756540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:33.757387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.757666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.757930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.759143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:33.760571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69e40-4a7e-458e-a981-348ea7f592e5 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:33.762265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 560/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:33 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:33.771418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:33.783972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:33.881162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b9028-ad4a-42ab-ae61-64aa7201e244 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:33.917281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:33.918649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:33.918906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:33.919311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04/users/e2e931531b734295bf0e94b1aeb8566f/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:33.923464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d532c007e2b42f68d221e03c815ac04, user_id=e2e931531b734295bf0e94b1aeb8566f, role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:33.925723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe4f3b-6b64-4b69-87e8-f5c0cf6f72a8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:33.944900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 563/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:54:33 2022] PUT /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04/users/e2e931531b734295bf0e94b1aeb8566f/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:54:35.925909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4084e0-2369-4ffc-b89c-c9a076ad13f5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:35.926864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4084e0-2369-4ffc-b89c-c9a076ad13f5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:35.926864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4084e0-2369-4ffc-b89c-c9a076ad13f5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:35.929071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 561/1124] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:35.935380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:35.936077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:35.936268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:35.956170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:35.957367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:36.026700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:36.032238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 564/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:36.057231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4084e0-2369-4ffc-b89c-c9a076ad13f5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:36.170544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4084e0-2369-4ffc-b89c-c9a076ad13f5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:36.207036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:36.207795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:36.208073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:36.208338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] PATH_INFO: `/v3/roles/9e4dd1adebe349ee813cb2c140fb937a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:36.209930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] RBAC: Authorizing `identity:get_role(role_id=9e4dd1adebe349ee813cb2c140fb937a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:36.211324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:36.213039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 562/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:36 2022] GET /identity/v3/roles/9e4dd1adebe349ee813cb2c140fb937a => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:36.221033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:36.334222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f770257-2613-4da9-98ca-97a0fecb0a1f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:36.371259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:36.372127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:36.372411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:36.372686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] PATH_INFO: `/v3/users/e2e931531b734295bf0e94b1aeb8566f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:36.374052 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] RBAC: Authorizing `identity:get_user(user_id=e2e931531b734295bf0e94b1aeb8566f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:36.375533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:36.377179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 565/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:36 2022] GET /identity/v3/users/e2e931531b734295bf0e94b1aeb8566f => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:36.383478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:36.429786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:36.485559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90e8c-0d1f-4ffc-ad84-61f01161a1a4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:36.520636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:36.521270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:36.521547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:36.521818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:36.596437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:36.597905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:36.645861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:36.706821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 563/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:36.714186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:36.789476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:36.825522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b1237-73ef-41c8-8e4a-d3f16c8a7658 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:36.860490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:36.861335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:36.861615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:36.861889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] PATH_INFO: `/v3/projects/3d532c007e2b42f68d221e03c815ac04` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:36.863121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] RBAC: Authorizing `identity:get_project(project_id=3d532c007e2b42f68d221e03c815ac04)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:36.864577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24212c42-7d54-463a-9d41-34e7c1773439 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:36.866258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 566/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:36 2022] GET /identity/v3/projects/3d532c007e2b42f68d221e03c815ac04 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:36.872613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:36.981951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ea788-2f1c-4a0d-ad57-a5131161e7ae None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:37.017525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:37.018157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:37.018464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:37.018739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:37.019555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] RBAC: Adding query filter params (role.id=9e4dd1adebe349ee813cb2c140fb937a, scope.project.id=3d532c007e2b42f68d221e03c815ac04, user.id=e2e931531b734295bf0e94b1aeb8566f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:37.019861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:37.021312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9fbfd-c61e-43c7-9ba4-c01cdc1325f7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:37.027216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 564/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:36 2022] GET /identity/v3/role_assignments?user.id=e2e931531b734295bf0e94b1aeb8566f&scope.project.id=3d532c007e2b42f68d221e03c815ac04&role.id=9e4dd1adebe349ee813cb2c140fb937a => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:39.054967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9164-e70c-455e-96ce-464696b94a4a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:39.055857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9164-e70c-455e-96ce-464696b94a4a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:39.055857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9164-e70c-455e-96ce-464696b94a4a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:39.058026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 567/1131] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:39.063488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:39.063924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:39.064035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:39.082032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:39.083231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:39.085295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:39.152750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 565/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:39.176438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9164-e70c-455e-96ce-464696b94a4a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:39.289123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9164-e70c-455e-96ce-464696b94a4a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:39.323294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:39.324014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:39.324293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:39.324562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:39.325779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:39.327372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98ff123-7179-40b0-9fc7-2045ef2981ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:39.328983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 568/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:39.335826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:39.455856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf8093-49e3-4444-bd2e-83a67f77e317 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:39.491388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:39.492040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:39.492320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:39.492594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:39.493302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:39.494771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f52221b-29e2-48be-914f-f42c49f9d573 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:39.503262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:39.523631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 566/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:54:39 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:41.564613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350930-5759-4183-82a0-eb1a41d5f33b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:41.565143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350930-5759-4183-82a0-eb1a41d5f33b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:41.565143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350930-5759-4183-82a0-eb1a41d5f33b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:41.566458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 569/1135] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:41.572753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:41.573461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:41.573613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:41.595841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:41.597623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:41.672019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 567/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:41.697921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350930-5759-4183-82a0-eb1a41d5f33b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:41.704316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:41.825444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350930-5759-4183-82a0-eb1a41d5f33b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:41.863684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:41.864462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:41.864742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:41.865011 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:41.866236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:41.867710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa412d-5c38-4f0a-98e3-50aced2553c5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:41.869300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 570/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:54:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:41.875496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:41.985090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9c3d9-3a68-4c28-9e9a-5abd6c4076e4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:42.007190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:42.025916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:42.026625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:42.026864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:42.027200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:42.027921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:42.029366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cbb4b-2a9a-4ff7-8dd8-95fb2046137a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:42.056588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 568/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:54:41 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:54:44.102179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99fda8-bd4b-4149-9542-fcfe891ce7de None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.102789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99fda8-bd4b-4149-9542-fcfe891ce7de None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.102789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99fda8-bd4b-4149-9542-fcfe891ce7de None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.104087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 571/1139] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:44.110404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.110854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.111043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.134170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:44.135381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:44.212272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 569/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:44.237695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99fda8-bd4b-4149-9542-fcfe891ce7de None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:44.319937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:44.352363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99fda8-bd4b-4149-9542-fcfe891ce7de None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:44.388646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:44.389449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.389727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.389969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.400964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:44.408585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:44.409698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:44.410743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 572/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:44.416060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:44.517853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97277d-f80d-451a-9746-8a3dbc4893b7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:44.525121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:44.554923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:44.555572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.555847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.556119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.556854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:44.557161 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:44.558552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:44.566872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 570/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:44.572867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:44.675236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b1866-56c2-4463-bc0e-05b4e1b3e2c5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:44.704289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:44.710236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:44.711128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.711422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.711670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] PATH_INFO: `/v3/users/2d8a7c3bea0a40be92b06a6111de3b47` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.725045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=2d8a7c3bea0a40be92b06a6111de3b47)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:44.726527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:44.728153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 573/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:44 2022] GET /identity/v3/users/2d8a7c3bea0a40be92b06a6111de3b47 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:54:44.736635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:44.848510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02ca18-6e3c-4471-8162-ea4d077ddd76 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:44.882778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:44.883399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:44.883675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:44.883944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:44.920438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:44.954587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:44.955946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:45.056172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 571/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:45.064360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:45.166564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828fe30d-6657-4565-a491-04ca33523ff6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:45.205639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:45.206498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:45.206779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:45.207051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] PATH_INFO: `/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:45.208314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] RBAC: Authorizing `identity:get_project(project_id=b4f1dc9c1d1c4eac800ae084d95b06f8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:45.209766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ef777-3488-436a-84ab-f38ad790a8f8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:45.211497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 574/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:45 2022] GET /identity/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:45.220449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:45.294188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:45.329387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3adf5-21e4-4948-b1dd-384ea08456ea None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:45.370044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:45.370719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:45.371001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:45.371273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:45.372093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=b4f1dc9c1d1c4eac800ae084d95b06f8, user.id=2d8a7c3bea0a40be92b06a6111de3b47) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:45.372403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:45.373866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f2dd7-4e2d-4d5e-8158-645f818c2fd4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:45.379618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 572/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:45 2022] GET /identity/v3/role_assignments?user.id=2d8a7c3bea0a40be92b06a6111de3b47&scope.project.id=b4f1dc9c1d1c4eac800ae084d95b06f8&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:47.345586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2a45c-b553-40e6-84a9-51dedd6c5b4b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:47.346152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2a45c-b553-40e6-84a9-51dedd6c5b4b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:47.346152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2a45c-b553-40e6-84a9-51dedd6c5b4b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:47.347520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 575/1147] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:47.353821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:47.354577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:47.354778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:47.380925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:47.382143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:47.452211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 573/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:47.475737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2a45c-b553-40e6-84a9-51dedd6c5b4b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:47.538342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:47.582508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2a45c-b553-40e6-84a9-51dedd6c5b4b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:47.618253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:47.619066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:47.619341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:47.619618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:47.630703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:47.638191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:47.639268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:47.640336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 576/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:47.647749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:47.710750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:47.751777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb7e74e-e337-4b35-b9ab-1281c24b5af3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:47.796750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:47.797356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:47.797611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:47.797912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:47.798695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:47.799000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:47.800370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:47.808762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 574/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:47.818311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:47.932937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101808-15be-46bb-a06f-70d3e85f6201 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:47.951429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:47.969519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:47.970422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:47.970700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:47.970968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] PATH_INFO: `/v3/users/2d8a7c3bea0a40be92b06a6111de3b47` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:47.972284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] RBAC: Authorizing `identity:get_user(user_id=2d8a7c3bea0a40be92b06a6111de3b47)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:47.973745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:47.975372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 577/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:47 2022] GET /identity/v3/users/2d8a7c3bea0a40be92b06a6111de3b47 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:47.981328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:48.083386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f33481-d996-473b-9160-801b52cb4f3f None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:48.106613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:48.118633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:48.119489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:48.119764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:48.120032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] PATH_INFO: `/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:48.121224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] RBAC: Authorizing `identity:get_project(project_id=b4f1dc9c1d1c4eac800ae084d95b06f8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:48.122678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e0e9-d939-42d3-87e7-cba134870c97 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:48.124518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 575/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:47 2022] GET /identity/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:48.130722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:48.241654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82160d1a-d141-406a-8818-7624c195aea1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:48.277249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:48.278132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:48.278443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:48.278724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] PATH_INFO: `/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8/users/2d8a7c3bea0a40be92b06a6111de3b47/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:48.280822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] RBAC: Authorizing `identity:create_grant(project_id=b4f1dc9c1d1c4eac800ae084d95b06f8, user_id=2d8a7c3bea0a40be92b06a6111de3b47, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:48.282266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2464f-4801-4d73-bdfa-6d7facfd551a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:48.299408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 578/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:54:48 2022] PUT /identity/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8/users/2d8a7c3bea0a40be92b06a6111de3b47/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:54:50.274456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168390-6d15-46a9-9694-14ef442caa16 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:50.274956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168390-6d15-46a9-9694-14ef442caa16 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:50.275047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168390-6d15-46a9-9694-14ef442caa16 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:50.277183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 576/1154] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:50.284654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:50.285066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:50.285244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:50.312712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:50.313916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:50.359243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:50.386749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 579/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:50.410091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168390-6d15-46a9-9694-14ef442caa16 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:50.529297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168390-6d15-46a9-9694-14ef442caa16 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:50.563393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:50.564167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:50.564442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:50.564709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:50.575242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:54:50.582757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:50.583788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:54:50.584633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 577/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:54:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:54:50.593545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:50.704800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e66ed5-1075-4815-a929-4e5c4f6947db None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:50.742691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:50.743347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:50.743626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:50.743903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:50.744641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:50.744949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:50.746352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:50.754374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 580/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:54:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:50.760357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:50.812291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:50.861046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ffcd9-3592-4892-bed3-3bbbba33f0d3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:50.895853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:50.896705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:50.896986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:50.897258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] PATH_INFO: `/v3/users/2d8a7c3bea0a40be92b06a6111de3b47` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:50.898508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] RBAC: Authorizing `identity:get_user(user_id=2d8a7c3bea0a40be92b06a6111de3b47)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:50.899933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:50.901543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 578/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:54:50 2022] GET /identity/v3/users/2d8a7c3bea0a40be92b06a6111de3b47 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:50.908301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:50.958134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:51.023167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cff64c-2f74-42fe-bf88-f9e76ae2a583 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:51.058366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:51.059018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:51.059299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:51.059568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:51.127571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:51.129017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:51.169213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:51.233479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 581/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:51.239764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:51.342012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a7c75-b42e-4551-98c4-18caa26537aa None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:51.363268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:51.385645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:51.386832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:51.387181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:51.387543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] PATH_INFO: `/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:51.389415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] RBAC: Authorizing `identity:get_project(project_id=b4f1dc9c1d1c4eac800ae084d95b06f8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:51.391175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03d62d-a88f-4d4d-8a56-ef541f59f390 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:51.392865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 579/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:54:51 2022] GET /identity/v3/projects/b4f1dc9c1d1c4eac800ae084d95b06f8 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:51.399287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:51.506970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b049e-2d71-4bfe-8273-5d90680b883a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:51.541635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:51.542275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:51.542591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:51.542862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:51.543683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=b4f1dc9c1d1c4eac800ae084d95b06f8, user.id=2d8a7c3bea0a40be92b06a6111de3b47) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:51.543995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:51.545431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d100ea6-760e-4779-8540-96ca17769202 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:51.551550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 582/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:54:51 2022] GET /identity/v3/role_assignments?user.id=2d8a7c3bea0a40be92b06a6111de3b47&scope.project.id=b4f1dc9c1d1c4eac800ae084d95b06f8&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:53.559828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790cd1-a47b-4ecf-8046-b011485c7aed None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:53.560375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790cd1-a47b-4ecf-8046-b011485c7aed None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:53.560375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790cd1-a47b-4ecf-8046-b011485c7aed None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:53.561770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 580/1162] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:53.568080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:53.568820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:53.569025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:53.577691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:53.599416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:53.601308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:53.673028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 583/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:53.706747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98790cd1-a47b-4ecf-8046-b011485c7aed None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:53.817416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98790cd1-a47b-4ecf-8046-b011485c7aed None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:53.852702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:53.853323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:53.853601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:53.853870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:53.909822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:53.930776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:53.933050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:54.035393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 581/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:54:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:54:54.044229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:54.166885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4ad33-b1b6-4b93-bf34-cd4d267071a5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:54.196755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:54.203223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:54.204030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:54.204311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:54.204580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:54.211486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:54.213276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fe83d914-4b4e-46a7-9603-5339776fa7a5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:54:54.227776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:54:54.228928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:54:54.229954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 584/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:54:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:54:54.235369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:54.322866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:54.336368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80421a2-4277-4068-b9b8-55694222f8ef None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:54.372235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:54.372878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:54.373155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:54.373424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:54.374165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:54:54.374492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:54.375851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b49582-9070-438f-badb-dd143366aa8b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:54.383152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 582/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Thu Jun 30 12:54:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:56.381785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c063-851c-451d-be18-a419813c19a7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:56.382671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c063-851c-451d-be18-a419813c19a7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:56.382671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906c063-851c-451d-be18-a419813c19a7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:56.384947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 585/1167] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:56.391252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874009-1cc5-4e06-ac90-423eb8352bf4 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:56.391954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874009-1cc5-4e06-ac90-423eb8352bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:56.392096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57874009-1cc5-4e06-ac90-423eb8352bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:56.414803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-57874009-1cc5-4e06-ac90-423eb8352bf4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:56.415999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-57874009-1cc5-4e06-ac90-423eb8352bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:56.494450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 583/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:56.517480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c063-851c-451d-be18-a419813c19a7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:56.627081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906c063-851c-451d-be18-a419813c19a7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:56.662634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:56.663303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:56.663564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:56.663837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:56.664564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:56.665938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa039ae-6f0f-44ae-886f-2df0c877c8fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:56.683076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 586/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Thu Jun 30 12:54:56 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:54:58.697808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a277f-ef2b-42f4-926e-7c1cede72310 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:58.698382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a277f-ef2b-42f4-926e-7c1cede72310 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:58.698382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a277f-ef2b-42f4-926e-7c1cede72310 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:58.699896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 584/1170] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:54:58.706408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:58.706870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:58.707046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:58.729747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:54:58.730983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:54:58.781424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:58.804475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 587/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:54:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:54:58.828807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a277f-ef2b-42f4-926e-7c1cede72310 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:58.926310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:54:58.947056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a277f-ef2b-42f4-926e-7c1cede72310 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:58.981640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:58.982465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:58.982747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:58.983169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] PATH_INFO: `/v3/domains/4a6e504649614a6b88a2b84a25709fdd` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:58.993504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a6e504649614a6b88a2b84a25709fdd)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:58.994983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96ea9e-73b4-40dc-b778-95bbfb16aba9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:58.996657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 585/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Thu Jun 30 12:54:58 2022] GET /identity/v3/domains/4a6e504649614a6b88a2b84a25709fdd => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:54:59.006104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:54:59.116713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2f8bf-24a3-4a9f-847c-2c168c6d7cc5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:54:59.154121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:54:59.154837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:54:59.155162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:54:59.155555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:54:59.156270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:54:59.157716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3ff74-d8fc-4b3d-8d82-0526104655ba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:54:59.187289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 588/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:54:59 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:55:01.209292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31921b-b7a9-44b8-95ad-b26ef76d24c8 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:01.209863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31921b-b7a9-44b8-95ad-b26ef76d24c8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:01.209863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31921b-b7a9-44b8-95ad-b26ef76d24c8 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:01.211257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 586/1174] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:55:01.218130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:01.218943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:01.219234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:01.244529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:55:01.245737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:55:01.276343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:01.320918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 589/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:55:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:55:01.347762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31921b-b7a9-44b8-95ad-b26ef76d24c8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:01.467854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31921b-b7a9-44b8-95ad-b26ef76d24c8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:01.501652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:01.502523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:01.502801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:01.503053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] PATH_INFO: `/v3/domains/4a6e504649614a6b88a2b84a25709fdd` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:01.504239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=4a6e504649614a6b88a2b84a25709fdd)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:01.505670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0957ab-92c8-4bab-bf15-189d606b32bf None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:01.507326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 587/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Thu Jun 30 12:55:01 2022] GET /identity/v3/domains/4a6e504649614a6b88a2b84a25709fdd => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:01.513732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:01.626599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289e985-542f-4b01-9b4d-32c3e8f18a58 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:01.675411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:01.676081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:01.676363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:01.676634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:01.677346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:01.678838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3f675-6d91-4548-8f94-26a8b3e5e904 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:01.706826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 590/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:55:01 2022] POST /identity/v3/users => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:55:03.827057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f0f9-84dd-420b-a57d-3fcb8ad30b51 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:03.827623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f0f9-84dd-420b-a57d-3fcb8ad30b51 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:03.827713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692f0f9-84dd-420b-a57d-3fcb8ad30b51 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:03.829115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 588/1178] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:55:03.835817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:03.836525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:03.836662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:03.858187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:55:03.859422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:55:03.875538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:03.931979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 591/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:55:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:55:03.955615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f0f9-84dd-420b-a57d-3fcb8ad30b51 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:03.969814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:04.084136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692f0f9-84dd-420b-a57d-3fcb8ad30b51 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:04.119432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:04.120213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:04.120494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:04.120752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:04.132491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:55:04.140133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:55:04.141182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:55:04.142121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 589/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:55:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:55:04.149473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:04.258965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5485-55dc-45b0-b5db-e569297bb5c0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:04.294142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:04.294843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:04.295121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:04.295394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:04.296378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:55:04.296701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:04.298093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:04.306341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 592/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:55:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:04.312981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:04.386914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:04.425516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1841-a827-4667-a4f6-944c132b7a96 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:04.461158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:04.462028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:04.462306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:04.462583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] PATH_INFO: `/v3/users/f9f6956c8e194f65b3830913c07adb7f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:04.476786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6956c8e194f65b3830913c07adb7f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:04.478228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:04.479927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 590/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:55:04 2022] GET /identity/v3/users/f9f6956c8e194f65b3830913c07adb7f => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:04.485984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:04.502606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:04.599523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070699a-1d0d-48c3-8f99-bd81dae6bb4a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:04.633669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:04.634305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:04.634613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:04.634886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:04.716639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:04.718305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:04.731818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:04.816381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 593/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:55:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:55:04.825923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:04.928996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fab573-f4bb-4b26-8545-35b0a5780c3d None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:04.955773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:04.968742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:04.969951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:04.970386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:04.970858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] PATH_INFO: `/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:04.972805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] RBAC: Authorizing `identity:get_project(project_id=158d46e0a27d4baa8d17af3ea46cd8b2)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:04.975148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f1b039-371e-4d54-9c29-8d2c77ca349b None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:04.977826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 591/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:55:04 2022] GET /identity/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:04.986178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:05.086774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161417e5-9642-4c11-be96-f08389904b39 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:05.100415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:05.123367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:05.124146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:05.124447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:05.124735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:05.125530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=158d46e0a27d4baa8d17af3ea46cd8b2, user.id=f9f6956c8e194f65b3830913c07adb7f) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:55:05.125836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:05.127297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774515ea-8f89-4cf7-90aa-b4d013f3d287 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:05.132916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 594/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:55:04 2022] GET /identity/v3/role_assignments?user.id=f9f6956c8e194f65b3830913c07adb7f&scope.project.id=158d46e0a27d4baa8d17af3ea46cd8b2&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:07.126654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6621-ae17-410c-9311-6e3ebf5240df None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.127233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6621-ae17-410c-9311-6e3ebf5240df None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.127233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a6621-ae17-410c-9311-6e3ebf5240df None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.129323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 592/1186] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:55:07.137857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.138262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.138488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.158986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:55:07.160165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:55:07.230895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 595/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:55:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:55:07.253461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a6621-ae17-410c-9311-6e3ebf5240df None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:07.358888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a6621-ae17-410c-9311-6e3ebf5240df None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:07.397217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:07.397924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.398198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.398494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.409255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:55:07.416760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:55:07.417905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:55:07.419041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 593/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:55:07.424717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:07.525456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bec66c-f99b-4388-9e05-9dc4f6220ba0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:07.533187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:07.561411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:07.562069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.562358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.562638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.563379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:55:07.563683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:07.565049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:07.573152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 596/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:07.579457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:07.642481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:07.687584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d6ec6-c32c-44c1-90e4-2830e3a80a18 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:07.722474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:07.723320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.723607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.723872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] PATH_INFO: `/v3/users/f9f6956c8e194f65b3830913c07adb7f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.725133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6956c8e194f65b3830913c07adb7f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:07.726573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:07.728212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 594/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:55:07 2022] GET /identity/v3/users/f9f6956c8e194f65b3830913c07adb7f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:07.735910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:07.845346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0700318c-6546-4383-9f95-2e04a07e3c6e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:07.880410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:07.881266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:07.881544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:07.881815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] PATH_INFO: `/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:07.883068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] RBAC: Authorizing `identity:get_project(project_id=158d46e0a27d4baa8d17af3ea46cd8b2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:07.884518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c82715-ee77-4d83-bc33-2a727302a1dd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:07.886230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 597/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:55:07 2022] GET /identity/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:07.892685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:08.000490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1fccc-4fb1-4da8-a2b5-f8e10023dd13 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:08.024988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:08.038921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:08.039802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:08.040079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:08.040353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] PATH_INFO: `/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2/users/f9f6956c8e194f65b3830913c07adb7f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:08.042598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] RBAC: Authorizing `identity:create_grant(project_id=158d46e0a27d4baa8d17af3ea46cd8b2, user_id=f9f6956c8e194f65b3830913c07adb7f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:08.044016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a6e77-f88e-4e5a-b434-7fdb1d6e6cfa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:08.058553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 595/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:55:07 2022] PUT /identity/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2/users/f9f6956c8e194f65b3830913c07adb7f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:55:10.058180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a08b-134c-4068-a62f-8265e0a62ebc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.059079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a08b-134c-4068-a62f-8265e0a62ebc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.059079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a08b-134c-4068-a62f-8265e0a62ebc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.061326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 598/1193] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:55:10.069458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.069871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.069977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.090759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:55:10.091958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:55:10.171705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 596/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:55:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:55:10.207783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3a08b-134c-4068-a62f-8265e0a62ebc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:10.260386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:10.332011 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3a08b-134c-4068-a62f-8265e0a62ebc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:10.367288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:10.368067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.368346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.368619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.378899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:55:10.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:55:10.387549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:55:10.388561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 599/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:55:10.400718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:10.520939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8a8a-12d4-4a6c-9f10-300351cbe1ae None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:10.535627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:10.566512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:10.567179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.567459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.567730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.568474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:55:10.568778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:10.570144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:10.578092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 597/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:10.584328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:10.688033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:10.694657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582348e9-94ce-440d-b87c-512c5a38b264 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:10.730559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:10.731431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.731709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.731980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] PATH_INFO: `/v3/users/f9f6956c8e194f65b3830913c07adb7f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.733375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] RBAC: Authorizing `identity:get_user(user_id=f9f6956c8e194f65b3830913c07adb7f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:10.734864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:10.736515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 600/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Thu Jun 30 12:55:10 2022] GET /identity/v3/users/f9f6956c8e194f65b3830913c07adb7f => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:10.742686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:10.867046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53e76-4ba2-4aae-98a1-c32ca42c9ea4 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:10.912211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:10.912848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:10.913128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:10.913399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:10.958333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:10.990469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:10.991920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:11.092005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 598/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:55:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:55:11.102520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:11.211018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f622f-8643-477d-a765-0a85afa8f32c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:11.249921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:11.250803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:11.251079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:11.251350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] PATH_INFO: `/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:11.252641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] RBAC: Authorizing `identity:get_project(project_id=158d46e0a27d4baa8d17af3ea46cd8b2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:11.254079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e4123d-e05d-49d1-8170-7783aee1526b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:11.255789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 601/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Thu Jun 30 12:55:11 2022] GET /identity/v3/projects/158d46e0a27d4baa8d17af3ea46cd8b2 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:11.262348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:55:11.344238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:11.371457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980d21d-cb35-4f72-9b1d-a006e7455bfd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:55:11.406314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:55:11.406981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:55:11.407256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:55:11.407525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:55:11.408328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=158d46e0a27d4baa8d17af3ea46cd8b2, user.id=f9f6956c8e194f65b3830913c07adb7f) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:55:11.408628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:55:11.410062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803e23c-3e05-4389-83b6-bfb4cf0fc50f None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:55:11.416340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 599/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:55:11 2022] GET /identity/v3/role_assignments?user.id=f9f6956c8e194f65b3830913c07adb7f&scope.project.id=158d46e0a27d4baa8d17af3ea46cd8b2&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:55:21.244711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:55:21.413242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=30 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:06.588336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8ab0-f87f-4c05-a698-1b12ff0d5b75 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:06.589313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8ab0-f87f-4c05-a698-1b12ff0d5b75 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:06.589313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d8ab0-f87f-4c05-a698-1b12ff0d5b75 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:06.590729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 602/1201] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:06.596514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:06.597045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:06.597045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:06.618406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:06.619653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:06.711608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 600/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:06.738406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d8ab0-f87f-4c05-a698-1b12ff0d5b75 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:06.870570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d8ab0-f87f-4c05-a698-1b12ff0d5b75 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:06.913685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:06.914540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:06.914825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:06.915098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:06.926805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:06.928297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:06.949089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:06.950233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:06.951345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:06.951345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:06.951345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:06.951345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:06.958551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 603/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:06.960000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:07.091952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bde4b-ee38-40b0-9fab-496d761e71e6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:07.125738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:07.132897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:07.133693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:07.134063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:07.134481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:07.135331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:07.136007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:07.137858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ae6403-4ea8-47a8-9384-27163bd570dd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:07.158745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 601/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:07.164770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:07.230181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:07.309216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac6e62-e617-4ba9-94c4-657ae6e1e54f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:07.351707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:07.352437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:07.352737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:07.353060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:07.353837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:07.355382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f427f8e-30c6-457e-bc1c-53a2d2fdda07 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:07.380813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 604/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Thu Jun 30 12:56:07 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 12:56:09.739026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2f1f3-5f7b-440d-9284-5d332d6e1b75 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:09.739622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2f1f3-5f7b-440d-9284-5d332d6e1b75 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:09.739622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff2f1f3-5f7b-440d-9284-5d332d6e1b75 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:09.741277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 602/1206] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:09.746911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:09.747368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:09.747588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:09.766046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:09.767386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:09.847509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 605/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:09.872049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2f1f3-5f7b-440d-9284-5d332d6e1b75 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:09.996826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff2f1f3-5f7b-440d-9284-5d332d6e1b75 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.039709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.040551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.040873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.041130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.055996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:10.064736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:10.066058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:10.066943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 603/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:56:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:10.072276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:10.119238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:10.194073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759faf0-0a12-4e81-b503-cd6bbe8764ca None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.234357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.234357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.234357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.234357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.235042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:10.235293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:10.236852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:10.253545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 606/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:56:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:10.255233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:10.318609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:10.373611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9474ed-1daa-4124-b428-a74bea4a0b78 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.414048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.414878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.415160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.415432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.427055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:10.428662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:10.451368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:10.452567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:10.453948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:10.453948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:10.453948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:10.453948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:10.453948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 604/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:10.458342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:10.577929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:10.581235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49273b-bbf4-41a6-b999-9c1d9381b475 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.618682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.619400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.619710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.619917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.620694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:10.621026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:10.622464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:10.635112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 607/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:10.641582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:10.758893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6371be-4b85-45aa-9074-8ab32e7e0cf1 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.762450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:10.798125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.798965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.799248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.799512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.807274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:10.808728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:10.823893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:10.825147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:10.826306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 605/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:10.833056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:10.947085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd31b0-15f7-4fae-af7a-62a06a71bb0c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:10.986971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:10.990497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:10.990497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:10.990497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:10.990497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:10.990497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:10.990885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:11.011875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 608/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:11.019277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:11.134656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d23541-6236-40d2-828b-d76baa638f60 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:11.172082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:11.172778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:11.173064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:11.173441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:11.207333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:11.252982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:11.254522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:11.374411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 606/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:56:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 12:56:11.375683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:11.397534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:11.485140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca801c6a-f30a-48f9-8cfa-9e1af12ec251 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:11.523534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:11.524376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:11.524745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:11.525036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:11.540724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:11.542193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:11.560791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:11.561924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:11.563381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:11.563381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:11.563381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:11.563381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:11.566882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 609/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:11.567872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:11.639819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:11.684375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d2873-82ae-4657-b977-afa14c5fa288 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:11.727610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:11.728277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:11.728548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:11.728818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:11.729545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:11.729848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:11.731242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:11.744805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 607/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:11.750811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:11.836661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:11.873353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd049cf-1bb3-4a15-926a-ff2bb9f8fc4b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:11.914519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:11.915332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:11.915616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:11.915895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:11.923510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:11.924939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:11.939081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:11.939987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:11.941470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 610/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:11.946180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:12.062612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ecdc7-d19d-4c02-a16f-2188fcb954c0 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:12.094718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:12.109792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:12.110495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:12.110759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:12.111032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:12.111807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:12.112126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:12.113581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6ec64-f54f-4b4f-82a9-85614e4387c6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:12.129006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 608/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:12.135133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:12.249383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c2fa4-8c39-49ec-91c8-4c45c990d1d0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:12.283998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:12.290112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:12.293352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:12.293352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:12.293352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:12.293352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=998f1e616d984e7fb7feac65a79c5599) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:12.293352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:12.295261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81de59-cdc7-4089-b923-4a261ed921a1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:12.304072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 611/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:56:12 2022] GET /identity/v3/role_assignments?user.id=998f1e616d984e7fb7feac65a79c5599&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:14.467430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa1a6-adf4-494b-b843-9d1765eb6ec5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:14.467968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa1a6-adf4-494b-b843-9d1765eb6ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:14.467968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029aa1a6-adf4-494b-b843-9d1765eb6ec5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:14.469338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 609/1220] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:14.477448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:14.478089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:14.478089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:14.497813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:14.499294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:14.574305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 612/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:14.601964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa1a6-adf4-494b-b843-9d1765eb6ec5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:14.710823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029aa1a6-adf4-494b-b843-9d1765eb6ec5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:14.749664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:14.750493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:14.750771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:14.751050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:14.764349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:14.773675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:14.774867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:14.775909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 610/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:56:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:14.781940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:14.886959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec20c3-e512-4490-8fd4-b624660ce477 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:14.914870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:14.927399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:14.928125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:14.928438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:14.928765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:14.929640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:14.929971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:14.931533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:14.942118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 613/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:56:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:14.948253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.004855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:15.072146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33435e9-483c-417a-ab22-22580196b957 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.109171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:15.109980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:15.110521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:15.110832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:15.122297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:15.124028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:15.142869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:15.143804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:15.144718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:15.144718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:15.144718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:15.144718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:15.144935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 611/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:15.151360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.263801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dfd2a5-a8de-4576-9b83-90fbc5d046ee None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.305789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:15.306587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:15.306892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:15.307168 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:15.308087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:15.308464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:15.310077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:15.322481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 614/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:15.328603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.432143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:15.440210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf245-1454-4f15-9929-601f4c4a438a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.476258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:15.477138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:15.477347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:15.477614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:15.485718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:15.487246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:15.501765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:15.502957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:15.504105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 612/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:56:15.511301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.561473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:15.620151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59855f9e-6cfc-4ca7-9700-5723a13f0b8e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.655321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:15.655979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:15.656283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:15.656533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:15.657369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:15.657681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:15.659189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:15.680165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 615/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:15.686679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.789477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b98cd8-6bac-4572-b0ce-e8629a250939 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.824265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:15.825088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:15.825366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:15.825637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:15.838466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:15.840414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:15.859267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:15.860610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:15.861560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:15.861560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:15.861560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:15.861560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:15.861560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 613/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:56:15.866758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:15.979946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdbf68f-c84a-48d3-8f87-8e4147169def None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:15.983044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:16.019532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:16.020214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:16.020497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:16.020777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:16.021497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:16.021808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:16.023239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:16.035134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 616/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:16.041091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:16.051652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:16.148669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe264d-c138-4707-9df9-ffb37fdaab23 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:16.181279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:16.182072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:16.182379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:16.182652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:16.190026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:16.191551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:16.205983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:16.207142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:16.208154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 614/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:16.213343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:16.324440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc557878-bf64-4c5e-a799-72a8d91c0a2e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:16.361990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:16.362687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:16.362967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:16.363238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:16.364022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:16.364326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:16.365866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67642d-efcd-4cb6-ac31-deb97d0518af None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:16.379679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 617/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:16.386117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:16.461374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:16.498839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fad2d3-d975-40cb-b649-fcfc4ef65452 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:16.534654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:16.535491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:16.535844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:16.536124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/998f1e616d984e7fb7feac65a79c5599/roles/4d222a92a96d465cb16adf1938871661` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:16.552573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=998f1e616d984e7fb7feac65a79c5599, role_id=4d222a92a96d465cb16adf1938871661)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:16.554214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae52bc-c7a3-4c04-9798-4f32d9a45d95 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:16.568190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 615/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:56:16 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/998f1e616d984e7fb7feac65a79c5599/roles/4d222a92a96d465cb16adf1938871661 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 12:56:18.759002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a576-0feb-47f7-8e0f-745965d804cc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:18.759588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a576-0feb-47f7-8e0f-745965d804cc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:18.759588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11a576-0feb-47f7-8e0f-745965d804cc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:18.761994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 618/1233] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:18.767659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:18.768111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:18.768201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:18.784834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:18.786166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:18.863612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 616/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:18.888680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11a576-0feb-47f7-8e0f-745965d804cc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:18.914775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:19.011827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11a576-0feb-47f7-8e0f-745965d804cc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.055091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.056008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.056388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.056667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.069420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:19.078568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:19.079789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:19.080952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 619/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Thu Jun 30 12:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:19.089563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:19.145264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:19.210502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3ecbc-bc36-428a-93df-19ba11f7f594 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.246498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.247162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.247446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.247809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.248654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:19.248953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:19.250364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:19.259332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 617/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:19.265390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:19.355097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:19.373046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22c171-f53d-4817-90a9-5860f82a19ad None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.411855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.412662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.412946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.413220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.424174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:19.425637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:19.443484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:19.444551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:19.445524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:19.445524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:19.445524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:19.445524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:19.445524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 620/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:19.451094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:19.559271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fb9d7-a82f-41fc-95e1-cf35794fa5d6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.569586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:19.605637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.606300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.606678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.607155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.607937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:19.608268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:19.609837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:19.627330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 618/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:19.628903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:19.733818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df803e54-b9bb-4c27-ba89-27eed019e592 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.768585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.769412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.769661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.769935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.772889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:19.778216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:19.780076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:19.794356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:19.795554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:19.796499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 621/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:19.801739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:19.915471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b9563-732a-4787-b9d8-ec0072a4e4b8 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:19.952540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:19.953230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:19.953435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:19.953797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:19.954601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:19.955071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:19.956621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:19.979339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 619/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:19.986281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:20.103996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04ff77-6c48-4a1b-a527-4513cfd9b43d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:20.143646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:20.144304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:20.144584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:20.145149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:20.208671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:20.215902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:20.217534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:20.333093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 622/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:56:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:56:20.334599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:20.364781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:20.447798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25618c89-39a1-4fd6-9de2-2e39f26854d3 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:20.484198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:20.484915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:20.485197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:20.485466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:20.496090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:20.497539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:20.515947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:20.517081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:20.517980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:20.517980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:20.517980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:20.517980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:20.517980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 620/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:20.523137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:20.621992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:20.630131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca67b9d-7588-48f5-a9c1-c59ff6ac35fa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:20.665159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:20.665822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:20.666096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:20.666413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:20.667110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:20.667410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:20.668793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:20.680314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 623/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:20.686557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:20.766038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:20.789130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6a32-051e-47b7-b72f-8e621eb162fe None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:20.827096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:20.827937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:20.828222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:20.828497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:20.835442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:20.836885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:20.851829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:20.852967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:20.853929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:20.858182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 621/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:20.859408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:20.972706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162cacb-b5a4-4147-9a90-b7b40aa20739 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:21.015477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:21.016401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:21.016680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:21.016964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:21.017869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:21.018250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:21.020148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab5fe2-fb13-42da-8ea6-518eb780da49 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:21.036136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 624/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:21.042463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:21.148769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966dc47-7d41-4b7e-8bcb-e56afe26ec89 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:21.183227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:21.183882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:21.184160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:21.184431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:21.185291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] RBAC: Adding query filter params (role.id=4d222a92a96d465cb16adf1938871661, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=998f1e616d984e7fb7feac65a79c5599) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:21.185598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:21.187050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa4e3a7-b983-4e39-a31c-2dd2d599f435 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:21.192973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 622/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:56:21 2022] GET /identity/v3/role_assignments?user.id=998f1e616d984e7fb7feac65a79c5599&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=4d222a92a96d465cb16adf1938871661 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:23.414553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0979-dde3-47e2-9793-1fe7638e1cb2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:23.415103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0979-dde3-47e2-9793-1fe7638e1cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:23.415103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0979-dde3-47e2-9793-1fe7638e1cb2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:23.416649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 625/1247] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:23.422668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:23.423134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:23.423236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:23.428603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:23.441872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:23.443209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:23.520317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 623/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 12:56:23.546680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0979-dde3-47e2-9793-1fe7638e1cb2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:23.566922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:23.659269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0979-dde3-47e2-9793-1fe7638e1cb2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:23.695038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:23.695797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:23.696097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:23.696349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:23.707382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:23.715833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:23.717149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:23.718094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 626/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:56:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 12:56:23.723224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:23.831402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f9676-49d5-4a24-991d-e7eb47eff648 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:23.866240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:23.872259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:23.873129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:23.873429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:23.873750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:23.874574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:23.874933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:23.876404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:23.885523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 624/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:56:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:23.891703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:23.982083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:24.011568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a501c6-b180-4120-832d-7ba5997b966d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:24.052398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:24.053326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:24.053677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:24.054054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:24.064972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:24.066694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:24.085004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:24.086038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:24.087139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:24.087139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:24.087139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:24.087139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:24.087139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 627/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:24.092567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:24.202358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800fa89-8597-4e75-9c28-63394a009726 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:24.238004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:24.238759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:24.238964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:24.239251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:24.240047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:24.240355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:24.241780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:24.254241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 625/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:24.260573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:24.371328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63efc4-ac65-469e-bf24-4afbf643384e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:24.403430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:24.412727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:24.413725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:24.414008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:24.414299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:24.422363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:24.423833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:24.440438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:24.441656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:24.442827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 628/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:24.447852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:24.507127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:24.554643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18425671-a69c-4b37-8da4-4a018f3ec0fd None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:24.595612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:24.596306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:24.596610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:24.596862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:24.597779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:24.598089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:24.599692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:24.618709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 626/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:24.625002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:24.736313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28c7f8-bfc7-43d8-ace0-55b614e516f3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:24.779945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:24.780780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:24.781225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:24.781684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:24.841092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:24.855088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:24.856657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:24.964311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 629/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:56:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:56:24.970699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:25.083023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f473eec-8749-41a5-a6da-951c1abb4751 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:25.097912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:25.120566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:25.121389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:25.121672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:25.121936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:25.134474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:25.135890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:25.154334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:25.155602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:25.156704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:25.156704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:25.156704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:25.156704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:25.156704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 627/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:25.161745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:25.259435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:25.277631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f000e2f-be42-4e9a-b623-a47ea5e75516 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:25.313924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:25.314612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:25.314894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:25.315167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:25.315961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:25.316227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:25.317655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:25.330314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 630/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:25.336403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:25.449996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d225af-2cf2-4d1d-a1c8-3c2bf8b8b2f9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:25.485567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:25.486374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:25.486651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:25.486941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:25.493891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:25.495352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:25.509754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:25.510965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:25.511915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 628/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:25.517521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:25.622883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da2262-51c7-42e1-a18d-0b570bcf66c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:25.660120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:25.660796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:25.661070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:25.661345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:25.662134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:25.662463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:25.663905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9a96d-9702-4bbf-a4bf-d11a2dd358c1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:25.681674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 631/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:25.683253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:25.711294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:25.796894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66895-7d96-42c3-ae90-614019a97e5e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:25.832320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:25.832979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:25.833265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:25.833539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:25.834414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=998f1e616d984e7fb7feac65a79c5599) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:25.834712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:25.836146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc569e-c7e9-49ac-8bf1-58a0525d38fa None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:25.842029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 629/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:56:25 2022] GET /identity/v3/role_assignments?user.id=998f1e616d984e7fb7feac65a79c5599&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:28.084689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee79ab-21bf-45f6-ac53-5a99c00ec47e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.085240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee79ab-21bf-45f6-ac53-5a99c00ec47e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.085240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee79ab-21bf-45f6-ac53-5a99c00ec47e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.086640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 632/1261] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:28.093276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.093726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.093803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.111223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:28.112416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:28.180986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 630/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:28.205411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee79ab-21bf-45f6-ac53-5a99c00ec47e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:28.217297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:28.316054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee79ab-21bf-45f6-ac53-5a99c00ec47e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:28.351212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:28.351921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.352193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.352461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.363185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:28.370688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:28.371809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:28.372841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 633/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:28.378377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:28.398467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:28.497179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708be72f-2eca-46f3-ad29-eb6b85c0f197 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:28.534008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:28.534696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.534977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.535245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.535989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:28.536297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:28.537656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:28.546758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 631/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:56:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:28.555521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:28.624346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:28.662683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca60af-3163-474d-b721-dac4ed254633 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:28.697146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:28.697921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.698196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.698493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.709599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:28.711086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:28.728328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:28.729348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:28.730245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:28.730245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:28.730245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:28.730245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:28.730245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 634/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:28.735978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:28.821575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:28.844564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8d73e-8096-4709-96bb-b6dd82ea61b9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:28.878918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:28.879497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:28.879776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:28.880047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:28.880756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:28.881064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:28.882441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:28.894503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 632/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:28.904089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.013700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cafdb-f8f7-4b2f-87ab-245da30c4815 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.039467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:29.054113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.054954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.055228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.055500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.063220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.064657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:29.079078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:29.080150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:29.081106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 635/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:29.086356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.188706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a4b4b-6f69-4315-ba01-cf3e4505df7e None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.224551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.225207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.225488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.225758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.226613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:29.226901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.228447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.237280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:29.246312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 633/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:29.252359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.369421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e84f-0bd6-403a-bfa3-c62b4614853b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.403607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.404528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.404809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.405078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.415949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.417397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:29.434858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:29.435937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:29.436834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:29.436834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:29.436834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:29.436834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:29.436834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 636/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:29.442265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.542513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d033d-a77d-4c79-b3e1-977e27386734 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.577603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.578236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.578564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.578858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.579569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:29.579870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.581225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.593219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 634/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:29.599758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.643358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:29.718224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559dc5f-8a4a-46bb-9277-f054eec24d53 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.755500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.756281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.756559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.756830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.764206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.765668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:29.780717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:29.781694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:29.782786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 637/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:29.788436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:29.860300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:29.904847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bd06d-3496-4a8e-8019-25af58829632 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:29.939179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:29.939816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:29.940095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:29.940358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:29.941126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:29.941429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:29.942875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea504fb4-2bae-4e59-a2cc-790c8ed9da8c None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:29.954967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 635/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:29.964811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:30.067725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:30.077821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c997f6-9f68-40df-9669-67208fcf5cb2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:30.117042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:30.117928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:30.118213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:30.118522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] PATH_INFO: `/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/998f1e616d984e7fb7feac65a79c5599/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:30.120740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] RBAC: Authorizing `identity:create_grant(project_id=a198bc016a3c49ae92f7ee4ae3dbd311, user_id=998f1e616d984e7fb7feac65a79c5599, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:30.122209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105016ba-c22c-4191-b994-20afcc9f7d09 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:30.140151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 638/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Thu Jun 30 12:56:29 2022] PUT /identity/v3/projects/a198bc016a3c49ae92f7ee4ae3dbd311/users/998f1e616d984e7fb7feac65a79c5599/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 12:56:32.156463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70f96f-f016-4e18-b6f6-8cee41f12449 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.157021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70f96f-f016-4e18-b6f6-8cee41f12449 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.157021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70f96f-f016-4e18-b6f6-8cee41f12449 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.158652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 636/1274] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:32.164709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.165390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.165621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.185054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:32.186495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:32.255617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 639/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:32.280431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70f96f-f016-4e18-b6f6-8cee41f12449 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:32.386594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70f96f-f016-4e18-b6f6-8cee41f12449 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:32.403999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:32.423366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:32.424085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.424365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.424634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.436338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 12:56:32.443893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:32.444954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:32.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 637/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Thu Jun 30 12:56:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:32.451695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:32.537759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:32.565987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9619b6-d760-42c9-8191-05bcba9bf551 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:32.601847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:32.602533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.602812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.603083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.603827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:32.604133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:32.605516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:32.613838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 640/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Thu Jun 30 12:56:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:32.623400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:32.730856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e87ef-83b8-4c6a-adef-62ba2a73bcce None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:32.772888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:32.773667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.773945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.774221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.785368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:32.786846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:32.804308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:32.805313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:32.811461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:32.811461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:32.811461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:32.811461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:32.811461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 638/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:32.811816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:32.949764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97322d6b-3c95-4910-88d7-561da5671b9d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:32.990530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:32.991193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:32.991693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:32.992042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:32.993315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:32.993949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:32.996092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:33.011075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 641/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:33.017149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:33.051655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:33.125204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19405837-84c4-4638-a595-a351df464dd9 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:33.162526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:33.163326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:33.163607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:33.163883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:33.171792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:33.173235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 12:56:33.187834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:33.188935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:33.190084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 639/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:33.197039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:33.201424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:33.307492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df43fe3-99a2-4d6a-8e10-3200f4e4bc1a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:33.346344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:33.347028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:33.347306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:33.347573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:33.348398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:33.348708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:33.350141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:33.367137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 642/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Thu Jun 30 12:56:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:33.373448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:33.470918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:33.487442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94525dd-9c90-4622-97f4-f65254854a7a None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:33.529611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:33.530266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:33.530576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:33.530851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:33.599728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:33.601162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:33.690775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:33.702462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 640/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Thu Jun 30 12:56:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 12:56:33.709218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:33.780434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:33.827690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f819b6-533f-4029-bb79-5fbc0b6b67c6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:33.863510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:33.864303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:33.864548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:33.864835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:33.875374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:33.876844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 12:56:33.895687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:33.896687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:33.897740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 12:56:33.897740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 12:56:33.897740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 12:56:33.897740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:33.897740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 643/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 12:56:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:33.903366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:34.017332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56476f86-15d5-4ad9-b2d0-e21a93b114fb None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:34.052699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:34.053274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:34.053547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:34.053820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:34.054553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:34.054862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:34.056222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:34.067685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 641/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Thu Jun 30 12:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:34.074007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:34.191244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f61ccb2-5eed-417d-9f0b-b48079aae076 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:34.206033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:34.228482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:34.229306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:34.229586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:34.229856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:34.236653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:34.238084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 12:56:34.251836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:34.252813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 12:56:34.253912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 30 12:56:34.257989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 644/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Thu Jun 30 12:56:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:34.259257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:34.309830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:34.372579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b818ad4-76fd-40be-b309-8bbbe4bb5e36 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:34.408151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:34.408781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:34.409058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:34.409336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:34.410109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:34.410444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:34.411884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddce16-55f9-46e5-98a9-a0ccc0807d5d None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:34.424564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 642/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Thu Jun 30 12:56:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:34.431632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:34.552766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682107ed-b3d6-4ce2-9d66-b948e338cb44 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:34.587276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:34.587850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:34.588126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:34.588392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:34.589559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] RBAC: Adding query filter params (role.id=2cbc0ba81e9c402a9daa6728ac06f33e, scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311, user.id=998f1e616d984e7fb7feac65a79c5599) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:34.589865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:34.591336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74c8a7-b260-4944-9aa5-e5397fe4d51b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:34.597399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 645/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Thu Jun 30 12:56:34 2022] GET /identity/v3/role_assignments?user.id=998f1e616d984e7fb7feac65a79c5599&scope.project.id=a198bc016a3c49ae92f7ee4ae3dbd311&role.id=2cbc0ba81e9c402a9daa6728ac06f33e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:36.629660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e728dd-314f-4222-8632-ee5250f4c10a None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:36.630302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e728dd-314f-4222-8632-ee5250f4c10a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:36.630302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e728dd-314f-4222-8632-ee5250f4c10a None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:36.632487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 643/1288] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:36.641174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:36.641597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:36.641731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:36.669118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:36.670670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:36.702119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:36.741389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 646/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:36.763909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e728dd-314f-4222-8632-ee5250f4c10a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:36.870592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e728dd-314f-4222-8632-ee5250f4c10a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:36.881289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:36.906487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:36.907271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:36.907543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:36.907811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:36.908453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:36.909810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:36.921755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:36.922990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:36.924736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 644/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:56:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:36.932478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:37.050592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e10b4-edb1-4c0a-b490-2d445eb6f20f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:37.088631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:37.089629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:37.089907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:37.090171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:37.090908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:37.091211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:37.092777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7308d-a42f-4c51-8264-06161fca88bd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:37.098077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 647/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Thu Jun 30 12:56:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:37.104407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:37.217978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee43b42-c153-4f4e-9962-1b147368d1a6 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:37.252545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:37.253223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:37.253500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:37.253772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] PATH_INFO: `/v3/services` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:37.254516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:37.254817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:37.256187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4738e-e81a-46aa-ab8c-d36bca7cd458 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:37.262414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 645/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Thu Jun 30 12:56:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:39.245174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319d8f-fa9a-4b48-80e9-319822a8a7be None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:39.246064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319d8f-fa9a-4b48-80e9-319822a8a7be None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:39.246064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319d8f-fa9a-4b48-80e9-319822a8a7be None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:39.248459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 648/1293] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:39.254616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b308-1f93-4297-a7cb-0472abf23abe None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:39.255077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b308-1f93-4297-a7cb-0472abf23abe None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:39.255181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793b308-1f93-4297-a7cb-0472abf23abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:39.274288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9793b308-1f93-4297-a7cb-0472abf23abe None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:39.275489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9793b308-1f93-4297-a7cb-0472abf23abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:39.307979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:39.346961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 646/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:39.371655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab319d8f-fa9a-4b48-80e9-319822a8a7be None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:39.418241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:39.500554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab319d8f-fa9a-4b48-80e9-319822a8a7be None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:39.535729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:39.536380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:39.536656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:39.536970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:39.537688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:39.539097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf864fbb-c937-496c-a441-11824670a3ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:39.556126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 649/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Thu Jun 30 12:56:39 2022] POST /identity/v3/services => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:56:41.581080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6b16-3821-4e99-ab42-5b653e102154 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:41.581614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6b16-3821-4e99-ab42-5b653e102154 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:41.581614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf6b16-3821-4e99-ab42-5b653e102154 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:41.582943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 647/1296] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:41.589413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:41.590117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:41.590303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:41.610560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:41.611750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:41.691901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 650/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:41.715301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6b16-3821-4e99-ab42-5b653e102154 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:41.843227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf6b16-3821-4e99-ab42-5b653e102154 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:41.893839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:41.894660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:41.894946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:41.895220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:41.895877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:41.897260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:41.909332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:41.910475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:41.911523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 648/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:56:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:41.916898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:41.936696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:42.027073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0687-0099-49a2-aa8b-c0ff43dee1b2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:42.064989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:42.065660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:42.065945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:42.066227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:42.066980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:42.067293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:42.068678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:42.074062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 651/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Thu Jun 30 12:56:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:42.080583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:42.113393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:42.185659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6106a-ea60-46fa-b934-b1a4cf542ac7 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:42.223416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:42.224061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:42.224342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:42.224616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:42.225404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] RBAC: Adding query filter params (interface=public, service_id=3c463739d0b140d580c1d47d5694a876, region_id=RegionOne) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:42.225716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:42.227113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea0ad2-746f-4444-a995-10f29382f4b3 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:42.234132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 649/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Thu Jun 30 12:56:42 2022] GET /identity/v3/endpoints?service_id=3c463739d0b140d580c1d47d5694a876&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 12:56:42.242513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:42.328610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:42.348075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d166e0f-9033-4de0-85d8-5134071b3ce0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:42.382371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:42.382964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:42.383238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:42.383514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:42.384188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:42.385550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5902a98-af28-4d00-8639-70158ba2f0cb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:42.392378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 652/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Thu Jun 30 12:56:42 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 12:56:44.401337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db14f9-c891-4a8a-abda-19ad0fd34e49 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:44.401928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db14f9-c891-4a8a-abda-19ad0fd34e49 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:44.401928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60db14f9-c891-4a8a-abda-19ad0fd34e49 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:44.403305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 650/1302] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:56:44.409825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:44.410644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:44.410805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:44.433743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:56:44.434993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:56:44.514511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 653/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:56:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 12:56:44.542224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db14f9-c891-4a8a-abda-19ad0fd34e49 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:44.647273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:44.655711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60db14f9-c891-4a8a-abda-19ad0fd34e49 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:44.700828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1929346-654a-4baf-92c4-285459737890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:44.702039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929346-654a-4baf-92c4-285459737890 None admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:44.702428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929346-654a-4baf-92c4-285459737890 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:44.702813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1929346-654a-4baf-92c4-285459737890 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:44.703801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1929346-654a-4baf-92c4-285459737890 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:44.705939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1929346-654a-4baf-92c4-285459737890 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application [None req-f1929346-654a-4baf-92c4-285459737890 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 12:56:44.722820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 12:56:44.723925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application with Lock( Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self._enter() Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application return self.creator() Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application created_value = creator( Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: ERROR keystone.server.flask.application Jun 30 12:56:44.724992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 651/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Thu Jun 30 12:56:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 12:56:44.730514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:44.810971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:44.835705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c67bfdb-8473-4315-9876-d6ff2465052e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:44.872067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:44.872750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:44.873140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:44.873416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:44.874125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 12:56:44.874469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:44.875866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90436f6e-5329-4219-87f6-cf7974cb3a0e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:44.881583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 654/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Thu Jun 30 12:56:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 12:56:44.888253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1929346-654a-4baf-92c4-285459737890 None admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:56:45.000918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1929346-654a-4baf-92c4-285459737890 None admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:56:45.034965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:56:45.035662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:56:45.035950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:56:45.036209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:56:45.036975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:56:45.038427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b252a5-b7c1-4cf9-a74f-0608c6aadb28 None admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:56:45.057601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 652/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Thu Jun 30 12:56:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 12:56:54.878491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=30 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:56:55.053252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:48.352828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9074-000e-4ae1-9f15-d0394317ef15 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:48.353431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9074-000e-4ae1-9f15-d0394317ef15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:48.353646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e9074-000e-4ae1-9f15-d0394317ef15 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:48.355232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 655/1307] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:48.361275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5f30b-2b6f-4b21-92de-2bffb3584506 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:48.361875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5f30b-2b6f-4b21-92de-2bffb3584506 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:48.362073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5f30b-2b6f-4b21-92de-2bffb3584506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:48.448902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-36c5f30b-2b6f-4b21-92de-2bffb3584506 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:48.449975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-36c5f30b-2b6f-4b21-92de-2bffb3584506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:48.561437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 653/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:57:48.612175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:48.612721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:48.613187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:48.615475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 656/1309] 10.222.0.49 () {54 vars in 964 bytes} [Thu Jun 30 12:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:48.621791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfffef-eaad-49fe-aac5-b6359a373d61 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:48.622265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfffef-eaad-49fe-aac5-b6359a373d61 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:48.622691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebfffef-eaad-49fe-aac5-b6359a373d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:48.769085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9ebfffef-eaad-49fe-aac5-b6359a373d61 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:48.772671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9ebfffef-eaad-49fe-aac5-b6359a373d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:48.821579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:48.920792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 654/1310] 10.222.0.49 () {58 vars in 1061 bytes} [Thu Jun 30 12:57:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:57:48.932053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:57:48.934346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:49.061689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ea3ed-2019-463b-9048-4f2a05c7718b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:57:49.147696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:57:49.148766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:49.149272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:49.149745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:49.151767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:49.213869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:49.284056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:57:49.285985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d6fcff-4a04-440b-9745-0b665dbefa09 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:57:49.307173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 657/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:57:51.661701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd27620-2fa8-4d1c-af83-84c868f2ce9d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:51.662265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd27620-2fa8-4d1c-af83-84c868f2ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:51.662265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd27620-2fa8-4d1c-af83-84c868f2ce9d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:51.663745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 655/1312] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:51.669124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc257-a572-4c22-a116-441de9c19b1a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:51.669569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc257-a572-4c22-a116-441de9c19b1a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:51.669741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfc257-a572-4c22-a116-441de9c19b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:51.741725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-14cfc257-a572-4c22-a116-441de9c19b1a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:51.743078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-14cfc257-a572-4c22-a116-441de9c19b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:51.850278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 658/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:57:51.901013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:51.901398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:51.901746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:51.903268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 656/1314] 10.222.0.49 () {54 vars in 964 bytes} [Thu Jun 30 12:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:51.910292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7bad-bccc-460a-a41a-85878bc23ee2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:51.910588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7bad-bccc-460a-a41a-85878bc23ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:51.910873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7bad-bccc-460a-a41a-85878bc23ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:51.923419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:51.996233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b62e7bad-bccc-460a-a41a-85878bc23ee2 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:51.997476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b62e7bad-bccc-460a-a41a-85878bc23ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:52.081017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 659/1315] 10.222.0.49 () {58 vars in 1061 bytes} [Thu Jun 30 12:57:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:57:52.090482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:57:52.092604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:52.199782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ce169-0b37-4f2e-94b5-3c2845a7662a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:57:52.280049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:52.283416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:57:52.284064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:52.284352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:52.284629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:52.287200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:52.400557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:57:52.402012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa40a-8a36-4a27-bb1b-1dc2e2f11232 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:57:52.420646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 657/1316] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:57:55.002195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:55.002816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:55.002816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:55.004821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 660/1317] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:57:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:55.012706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebb54-1727-4eb7-a573-0c106a5f97a6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:55.013171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebb54-1727-4eb7-a573-0c106a5f97a6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:55.013277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eebb54-1727-4eb7-a573-0c106a5f97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:55.078853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-88eebb54-1727-4eb7-a573-0c106a5f97a6 None None] MFA Rules not processed for user `63b8e8615a52438eba7871052809289f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:55.080079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-88eebb54-1727-4eb7-a573-0c106a5f97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:55.147222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:55.197467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 658/1318] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:57:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:57:55.241259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:57:55.243779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:55.357005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2eb416-cc6d-4634-955b-6d7ff34cc31c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:57:55.360041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:55.435289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:57:55.435958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:55.436238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:55.436508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:55.438194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:55.559116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:57:55.560502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbf16b9-1203-46da-a3a3-f00951779cb0 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:57:55.579470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 661/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:57:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:57:58.307976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:58.308418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:58.308512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:58.310273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 659/1320] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:57:58.318775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cadef-f663-481b-b8cf-99827936e1d8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:58.319224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cadef-f663-481b-b8cf-99827936e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:58.319463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cadef-f663-481b-b8cf-99827936e1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:58.363997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:58.391435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a36cadef-f663-481b-b8cf-99827936e1d8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:57:58.392648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a36cadef-f663-481b-b8cf-99827936e1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:58.502178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 662/1321] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:57:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:57:58.544982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:57:58.547337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:58.664757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58f410-a217-460b-b0f8-c516425efca1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:57:58.742564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:57:58.743207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:57:58.743486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:57:58.743761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:57:58.745416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:57:58.753283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:57:58.866949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:57:58.868392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d169dcf-867c-4513-b450-4ec11a0786d4 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:57:58.892205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 660/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:57:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:01.605394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868540-5085-4a39-8a50-b907402b159a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:01.605908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868540-5085-4a39-8a50-b907402b159a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:01.605908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4868540-5085-4a39-8a50-b907402b159a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:01.607417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 663/1323] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:01.612953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdfbbb-9a2a-4446-9ec6-6fd687e8d62f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:01.613382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdfbbb-9a2a-4446-9ec6-6fd687e8d62f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:01.613578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdfbbb-9a2a-4446-9ec6-6fd687e8d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:01.680022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e2bdfbbb-9a2a-4446-9ec6-6fd687e8d62f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:01.681256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e2bdfbbb-9a2a-4446-9ec6-6fd687e8d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:01.744352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:01.788852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 661/1324] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:01.830700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4868540-5085-4a39-8a50-b907402b159a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:01.832395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b4868540-5085-4a39-8a50-b907402b159a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:01.948691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4868540-5085-4a39-8a50-b907402b159a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:01.961552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:02.027953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:02.028603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:02.028879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:02.029154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:02.030896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:02.140483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:02.142458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e7ab7-ad99-4101-b922-af83498f760b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:02.164618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 664/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:04.268180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:04.268702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:04.268823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:04.270238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 662/1326] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:04.276179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ea8fd-362f-47c6-8ba6-e8bcba2bb7eb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:04.276624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ea8fd-362f-47c6-8ba6-e8bcba2bb7eb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:04.276624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ea8fd-362f-47c6-8ba6-e8bcba2bb7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:04.328430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:04.347651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5a3ea8fd-362f-47c6-8ba6-e8bcba2bb7eb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:04.348875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5a3ea8fd-362f-47c6-8ba6-e8bcba2bb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:04.448683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 665/1327] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:04.485012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:04.486597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:04.594263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029978c-e9eb-486d-9951-72d0288acd81 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:04.671942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:04.672619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:04.672901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:04.673175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:04.674996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:04.695432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:04.793443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:04.794962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51323-4875-46c3-8a60-43a2e36bbb21 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:04.814168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 663/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:06.872910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:06.873460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:06.873460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:06.875081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 666/1329] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:06.880244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403398-1933-4167-8df7-c51e6da68363 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:06.880691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403398-1933-4167-8df7-c51e6da68363 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:06.880800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e403398-1933-4167-8df7-c51e6da68363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:06.948819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7e403398-1933-4167-8df7-c51e6da68363 None None] MFA Rules not processed for user `63b8e8615a52438eba7871052809289f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:06.950048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7e403398-1933-4167-8df7-c51e6da68363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:07.020734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:07.054031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 664/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:07.092257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:07.093869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:07.204060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282586f9-7584-43c9-b547-e41bc1b93f2b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:07.225560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:07.282351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:07.283012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:07.283266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:07.283561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:07.285259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:07.396714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:07.398178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8eb59a-7836-4cad-a6d8-fc90b66b9883 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:07.416350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 667/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:09.705246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36148-f628-42ae-9843-18073243e102 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:09.706237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36148-f628-42ae-9843-18073243e102 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:09.706237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f36148-f628-42ae-9843-18073243e102 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:09.708465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 665/1332] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:09.714978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5c3ab-92e3-4e6c-817d-9d37e015260d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:09.716128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5c3ab-92e3-4e6c-817d-9d37e015260d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:09.716305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5c3ab-92e3-4e6c-817d-9d37e015260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:09.787733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:09.790588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c9f5c3ab-92e3-4e6c-817d-9d37e015260d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:09.792251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9f5c3ab-92e3-4e6c-817d-9d37e015260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:09.893449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 668/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:09.941086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36148-f628-42ae-9843-18073243e102 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:09.942615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-90f36148-f628-42ae-9843-18073243e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:10.052864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f36148-f628-42ae-9843-18073243e102 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:10.133366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:10.134019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:10.134297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:10.134609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:10.136286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:10.165529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:10.265767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:10.267273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd13f5-2ab6-46bb-b609-0be10a7252cd service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:10.285660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 666/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:12.353984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:12.354586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:12.354586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:12.355920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 669/1335] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:12.362881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ed38-b65d-415b-9496-aee07d17b0eb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:12.363648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ed38-b65d-415b-9496-aee07d17b0eb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:12.363801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318ed38-b65d-415b-9496-aee07d17b0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:12.441888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0318ed38-b65d-415b-9496-aee07d17b0eb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:12.443155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0318ed38-b65d-415b-9496-aee07d17b0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:12.520213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:12.543135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 667/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:12.587779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:12.589670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:12.701800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4040cd4-3457-49a8-bde4-3d277381120e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:12.732012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:12.780973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:12.781532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:12.781803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:12.782071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:12.783740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:12.905380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:12.906981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18b3e8-c92f-448c-8d4b-10fd3948438f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:12.925926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 670/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:58:15.424824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:15.425715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:15.425715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:15.428175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 668/1338] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:15.434582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdacb83-c77f-4dec-a590-5425b25f59bd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:15.435028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdacb83-c77f-4dec-a590-5425b25f59bd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:15.435132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdacb83-c77f-4dec-a590-5425b25f59bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:15.505738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1bdacb83-c77f-4dec-a590-5425b25f59bd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:15.506992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1bdacb83-c77f-4dec-a590-5425b25f59bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:15.515426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:15.610473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 671/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:15.650031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:15.652729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:15.779233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdcbb3-8b76-4222-83fb-7f052f78fde1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:15.862722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:15.863654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:15.864022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:15.864273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:15.866984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:15.913961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:15.992855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:15.994485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdf71b-b983-44dc-8468-2a3617adb339 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:16.014622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 669/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:20.154471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:20.155022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:20.155022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:20.156412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 672/1341] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:20.162412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7fbda-db1a-480d-9cbb-2f71357e4df7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:20.163205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7fbda-db1a-480d-9cbb-2f71357e4df7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:20.163647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb7fbda-db1a-480d-9cbb-2f71357e4df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:20.240563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-1bb7fbda-db1a-480d-9cbb-2f71357e4df7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:20.241775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1bb7fbda-db1a-480d-9cbb-2f71357e4df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:20.341198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:20.355733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 670/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:20.395509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:20.397849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:20.512194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62f16e-6ddd-49ad-a3a0-4db0b66108fc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:20.516518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:20.517237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:20.517513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:20.517782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:20.527723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:20.529163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa84b4-baa5-4f12-ad99-fe00c7f66c9c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:20.539426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 673/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:23.160622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:23.161194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:23.161194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:23.162812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 671/1344] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:23.168384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab1d0-82e1-4749-a94c-f4babc1f0542 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:23.168818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab1d0-82e1-4749-a94c-f4babc1f0542 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:23.168910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab1d0-82e1-4749-a94c-f4babc1f0542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:23.200539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5d3ab1d0-82e1-4749-a94c-f4babc1f0542 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:23.202093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5d3ab1d0-82e1-4749-a94c-f4babc1f0542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:23.235590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:23.293092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 674/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:23.331557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:23.333418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:23.347637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb70e6-08f9-4d69-af9c-219b92e7738c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:23.351304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:23.351950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:23.352228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:23.352491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:23.362810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:23.364245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c3957-aba3-40f0-b527-8b8e61bddcd3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:23.373658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 672/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:27.370797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93387-04b0-48cc-bda9-483b9b381533 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:27.371355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93387-04b0-48cc-bda9-483b9b381533 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:27.371355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f93387-04b0-48cc-bda9-483b9b381533 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:27.372729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 675/1347] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:27.378765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db93c-8fe6-4910-8b0e-7f39844a4a6d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:27.379221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db93c-8fe6-4910-8b0e-7f39844a4a6d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:27.379332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db93c-8fe6-4910-8b0e-7f39844a4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:27.398179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-736db93c-8fe6-4910-8b0e-7f39844a4a6d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:27.399429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-736db93c-8fe6-4910-8b0e-7f39844a4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:27.407989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 673/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:27.446422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f93387-04b0-48cc-bda9-483b9b381533 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:27.459234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f93387-04b0-48cc-bda9-483b9b381533 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:27.463357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:27.463992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:27.464269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:27.464555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:27.475756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:27.477383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22de14-12a9-45ad-8b2d-8b6d7ac7b6d7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:27.487799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 676/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:32.476251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c082-eb17-4ee6-badc-ca9aeb885224 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:32.476800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c082-eb17-4ee6-badc-ca9aeb885224 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:32.476800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c082-eb17-4ee6-badc-ca9aeb885224 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:32.486880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 674/1350] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 12:58:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:32.492227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cb3f8-b653-4604-8d3d-44f155797fd8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:32.492709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cb3f8-b653-4604-8d3d-44f155797fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:32.492791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78cb3f8-b653-4604-8d3d-44f155797fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:32.563934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b78cb3f8-b653-4604-8d3d-44f155797fd8 None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:32.566154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b78cb3f8-b653-4604-8d3d-44f155797fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:32.646111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 677/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 12:58:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:36.689483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada9c8-cf9a-42fc-b958-a08ccab2939f None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:36.690610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada9c8-cf9a-42fc-b958-a08ccab2939f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:36.690996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ada9c8-cf9a-42fc-b958-a08ccab2939f None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:36.692428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 675/1352] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 12:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:36.697630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965cd0-db40-437e-a0c3-7ee38dbee26a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:36.698075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965cd0-db40-437e-a0c3-7ee38dbee26a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:36.698179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f965cd0-db40-437e-a0c3-7ee38dbee26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:36.717523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0f965cd0-db40-437e-a0c3-7ee38dbee26a None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:36.718801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f965cd0-db40-437e-a0c3-7ee38dbee26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:36.728085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 678/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:37.393991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=13 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:38.277470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3e46-967a-4f7a-b73a-ca12cfcbd2e0 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:38.278119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3e46-967a-4f7a-b73a-ca12cfcbd2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:38.278119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be3e46-967a-4f7a-b73a-ca12cfcbd2e0 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:38.279419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 676/1354] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 12:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:38.284891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2044-1e33-44a5-b4bf-3f097abe2da0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:38.285310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2044-1e33-44a5-b4bf-3f097abe2da0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:38.285506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa2044-1e33-44a5-b4bf-3f097abe2da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:38.305080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3caa2044-1e33-44a5-b4bf-3f097abe2da0 None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:38.306294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3caa2044-1e33-44a5-b4bf-3f097abe2da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:38.316050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 679/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 12:58:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:41.826943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b917-85a4-4914-b132-27299d9d8e36 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:41.827524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b917-85a4-4914-b132-27299d9d8e36 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:41.827524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2b917-85a4-4914-b132-27299d9d8e36 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:41.834646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 677/1356] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:41.835672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c85f4-33ef-401c-844d-a3211fa3d5d9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:41.836130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c85f4-33ef-401c-844d-a3211fa3d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:41.837110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c85f4-33ef-401c-844d-a3211fa3d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:41.861896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-292c85f4-33ef-401c-844d-a3211fa3d5d9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:41.863410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-292c85f4-33ef-401c-844d-a3211fa3d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:41.878467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 680/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:41.941616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96cad6-0bf4-4c0a-9774-cdd24798717b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:41.941938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96cad6-0bf4-4c0a-9774-cdd24798717b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:41.942172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96cad6-0bf4-4c0a-9774-cdd24798717b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:41.944369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 678/1358] 10.222.0.49 () {54 vars in 961 bytes} [Thu Jun 30 12:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:41.954671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a595f59-8566-403f-bf9f-96a8a979de42 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:41.955091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a595f59-8566-403f-bf9f-96a8a979de42 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:41.955468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a595f59-8566-403f-bf9f-96a8a979de42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:42.006494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:42.014660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3a595f59-8566-403f-bf9f-96a8a979de42 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:42.016196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a595f59-8566-403f-bf9f-96a8a979de42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:42.100428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 681/1359] 10.222.0.49 () {58 vars in 1058 bytes} [Thu Jun 30 12:58:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:42.108669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96cad6-0bf4-4c0a-9774-cdd24798717b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:42.124795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96cad6-0bf4-4c0a-9774-cdd24798717b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:42.130464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:42.131116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:42.131450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:42.131732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:42.140939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:42.142606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a40f7-807a-4d23-92cb-438ee4c9e449 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:42.152487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 679/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:44.381199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b4c-ab05-4c81-ba58-765744e5ba18 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:44.381736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b4c-ab05-4c81-ba58-765744e5ba18 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:44.381736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b4c-ab05-4c81-ba58-765744e5ba18 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:44.384342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 682/1361] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:44.388451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e4196-1d9d-4112-a1e3-8a17979d4b20 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:44.388915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e4196-1d9d-4112-a1e3-8a17979d4b20 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:44.389023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e4196-1d9d-4112-a1e3-8a17979d4b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:44.407822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b65e4196-1d9d-4112-a1e3-8a17979d4b20 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:44.409045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b65e4196-1d9d-4112-a1e3-8a17979d4b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:44.417409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 680/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:44.461948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0b4c-ab05-4c81-ba58-765744e5ba18 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:44.477546 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0b4c-ab05-4c81-ba58-765744e5ba18 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:44.483396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:44.484229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:44.484793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:44.485299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:44.494926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:44.496740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06baa4e5-5ffe-4047-9fed-6bfd51fc13ce service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:44.507770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 683/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:46.823775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4291-8c03-4bf7-b655-e128557bb263 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:46.824218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4291-8c03-4bf7-b655-e128557bb263 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:46.824770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd4291-8c03-4bf7-b655-e128557bb263 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:46.825864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 681/1364] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:46.831494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141b41-7f28-47ac-816b-d5d6c12376ce None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:46.831963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141b41-7f28-47ac-816b-d5d6c12376ce None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:46.832046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a141b41-7f28-47ac-816b-d5d6c12376ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:46.850682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6a141b41-7f28-47ac-816b-d5d6c12376ce None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:46.851890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6a141b41-7f28-47ac-816b-d5d6c12376ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:46.860058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 684/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:46.917857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a21cc-2fa3-4f4a-ad92-d5ab7a2f9845 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:46.918216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a21cc-2fa3-4f4a-ad92-d5ab7a2f9845 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:46.918459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a21cc-2fa3-4f4a-ad92-d5ab7a2f9845 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:46.919979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 682/1366] 10.222.0.49 () {54 vars in 961 bytes} [Thu Jun 30 12:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:46.925982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9613a-d0a7-4c98-9b3a-9a8ab7854d43 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:46.926292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9613a-d0a7-4c98-9b3a-9a8ab7854d43 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:46.926603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9613a-d0a7-4c98-9b3a-9a8ab7854d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:46.947737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7ab9613a-d0a7-4c98-9b3a-9a8ab7854d43 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:46.948867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7ab9613a-d0a7-4c98-9b3a-9a8ab7854d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:46.958124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 685/1367] 10.222.0.49 () {58 vars in 1058 bytes} [Thu Jun 30 12:58:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:46.966387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a21cc-2fa3-4f4a-ad92-d5ab7a2f9845 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:46.980859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a21cc-2fa3-4f4a-ad92-d5ab7a2f9845 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:46.986742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:46.987503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:46.987715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:46.988220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:46.996705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:46.998291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d1fd5-5975-433e-a70b-d167b99553b3 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:47.007841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 683/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:58:49.117521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13600-fe42-4755-ac94-ee6a4eb2e7b0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:49.118576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13600-fe42-4755-ac94-ee6a4eb2e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:49.119087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a13600-fe42-4755-ac94-ee6a4eb2e7b0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:49.120674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 686/1369] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:49.126051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099d1c-52b7-4b7a-ac89-8db311f6c1bf None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:49.126903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099d1c-52b7-4b7a-ac89-8db311f6c1bf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:49.127385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61099d1c-52b7-4b7a-ac89-8db311f6c1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:49.145763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-61099d1c-52b7-4b7a-ac89-8db311f6c1bf None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:49.147307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-61099d1c-52b7-4b7a-ac89-8db311f6c1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:49.155473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 684/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:49.202154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a13600-fe42-4755-ac94-ee6a4eb2e7b0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:49.218408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a13600-fe42-4755-ac94-ee6a4eb2e7b0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:49.224731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:49.225604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:49.226083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:49.226564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:49.235702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:49.237517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0a653-f815-4306-92b9-4763fa5efe13 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:49.247493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 687/1371] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:49.883000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436761-5669-4201-ac2c-404645e6dd7d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:49.883430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436761-5669-4201-ac2c-404645e6dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:49.883608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436761-5669-4201-ac2c-404645e6dd7d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:49.884994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 685/1372] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:49.890431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b76eb-7f27-4690-a280-be2602b182a4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:49.890708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b76eb-7f27-4690-a280-be2602b182a4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:49.891176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b76eb-7f27-4690-a280-be2602b182a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:49.909725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3f7b76eb-7f27-4690-a280-be2602b182a4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:49.910983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f7b76eb-7f27-4690-a280-be2602b182a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:49.919303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 688/1373] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:50.262485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d7bf-7b6e-4079-978a-c45562021b36 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:50.262485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d7bf-7b6e-4079-978a-c45562021b36 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:50.262485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d56d7bf-7b6e-4079-978a-c45562021b36 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:50.262485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 686/1374] 10.222.0.49 () {54 vars in 963 bytes} [Thu Jun 30 12:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:50.268682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7c86e-1289-44f9-8b9d-b2847076d94d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:50.268682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7c86e-1289-44f9-8b9d-b2847076d94d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:50.268682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7c86e-1289-44f9-8b9d-b2847076d94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:50.320296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ddd7c86e-1289-44f9-8b9d-b2847076d94d None None] MFA Rules not processed for user `607bae995849489db891582db395e28c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:50.321500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ddd7c86e-1289-44f9-8b9d-b2847076d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:50.392219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 689/1375] 10.222.0.49 () {58 vars in 1060 bytes} [Thu Jun 30 12:58:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:58:50.400186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56d7bf-7b6e-4079-978a-c45562021b36 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:50.411470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d56d7bf-7b6e-4079-978a-c45562021b36 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:50.415187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:50.415894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:50.416239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:50.416578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:50.426494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:50.428177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2974ca6-6146-4a3b-ac12-786857edc80f service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:50.439179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 687/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:51.292690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774872e7-43e8-4f35-9721-4fd891e80c85 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:51.293253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774872e7-43e8-4f35-9721-4fd891e80c85 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:51.293253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774872e7-43e8-4f35-9721-4fd891e80c85 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:51.294626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 690/1377] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:51.300202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760f142-91fe-4e41-aa7f-1a65ca885407 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:51.300654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760f142-91fe-4e41-aa7f-1a65ca885407 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:51.300889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760f142-91fe-4e41-aa7f-1a65ca885407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:51.319208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3760f142-91fe-4e41-aa7f-1a65ca885407 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:51.320428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3760f142-91fe-4e41-aa7f-1a65ca885407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:51.329377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 688/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:51.373913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774872e7-43e8-4f35-9721-4fd891e80c85 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:51.389549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774872e7-43e8-4f35-9721-4fd891e80c85 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:51.395471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:51.396107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:51.396389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:51.396658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:51.404990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:51.406721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dc728-5311-4d4d-8af1-b645097413d9 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:51.410615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:58:51.417322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 691/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:53.464753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8911f-c1de-4f21-a585-7bfde97c3d8b None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:53.465603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8911f-c1de-4f21-a585-7bfde97c3d8b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:53.466420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8911f-c1de-4f21-a585-7bfde97c3d8b None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:53.468295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 689/1380] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:53.474162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0474-1e42-4a3e-b5b0-d597c58ca060 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:53.474664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0474-1e42-4a3e-b5b0-d597c58ca060 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:53.474962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0474-1e42-4a3e-b5b0-d597c58ca060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:53.494791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d1ba0474-1e42-4a3e-b5b0-d597c58ca060 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:53.496151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d1ba0474-1e42-4a3e-b5b0-d597c58ca060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:53.505158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 692/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:53.554539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8911f-c1de-4f21-a585-7bfde97c3d8b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:53.570671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8911f-c1de-4f21-a585-7bfde97c3d8b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:53.577831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:53.578617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:53.579076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:53.579356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:53.589049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:53.591367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa3fb-f9fd-4a88-ba7a-7e67aac8ffbf service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:53.602769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 690/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:55.694145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bf5-1511-4001-a847-67a2025e1aa3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:55.694773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bf5-1511-4001-a847-67a2025e1aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:55.694773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942bf5-1511-4001-a847-67a2025e1aa3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:55.696295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 693/1383] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:55.701387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadef17-8c63-4dc1-a00b-1107052ef389 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:55.701825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadef17-8c63-4dc1-a00b-1107052ef389 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:55.701941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadef17-8c63-4dc1-a00b-1107052ef389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:55.720450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2eadef17-8c63-4dc1-a00b-1107052ef389 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:55.721635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2eadef17-8c63-4dc1-a00b-1107052ef389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:55.729917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 691/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:55.771238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58942bf5-1511-4001-a847-67a2025e1aa3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:55.787055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58942bf5-1511-4001-a847-67a2025e1aa3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:55.792474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:55.793121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:55.793401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:55.793637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:55.806776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:55.808521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b380d4-7f8d-451b-a6ff-510c4bcdb57f service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:55.819494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 694/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:58:57.876455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dfa4-6ed5-4230-a307-9eb843b1394e None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:57.877057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dfa4-6ed5-4230-a307-9eb843b1394e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:57.877057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253dfa4-6ed5-4230-a307-9eb843b1394e None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:57.878402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 692/1386] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:58:57.883776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd88a-2582-4811-82b4-200202cefefa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:57.884550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd88a-2582-4811-82b4-200202cefefa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:57.884829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fd88a-2582-4811-82b4-200202cefefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:57.904404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9b6fd88a-2582-4811-82b4-200202cefefa None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:58:57.905636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9b6fd88a-2582-4811-82b4-200202cefefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:58:57.914601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 695/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:58:57.958707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253dfa4-6ed5-4230-a307-9eb843b1394e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:58:57.974764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253dfa4-6ed5-4230-a307-9eb843b1394e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:58:57.981798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:58:57.982723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:58:57.983063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:58:57.983426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:58:57.994981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:58:57.997308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d38d03-2887-4ba6-b13e-3a1d4c970f59 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:58:58.011569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 693/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:00.615293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005e43-8a2e-4a05-9e6a-48ced99234c4 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:00.615867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005e43-8a2e-4a05-9e6a-48ced99234c4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:00.615867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d005e43-8a2e-4a05-9e6a-48ced99234c4 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:00.617190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 696/1389] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:00.624433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5e51e-e212-4d98-966d-5c41f5548fee None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:00.624887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5e51e-e212-4d98-966d-5c41f5548fee None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:00.625000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5e51e-e212-4d98-966d-5c41f5548fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:00.643436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d3a5e51e-e212-4d98-966d-5c41f5548fee None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:00.644667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d3a5e51e-e212-4d98-966d-5c41f5548fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:00.653539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 694/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:00.698433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d005e43-8a2e-4a05-9e6a-48ced99234c4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:00.713952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d005e43-8a2e-4a05-9e6a-48ced99234c4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:00.720929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:00.721583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:00.721878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:00.722133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:00.731286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:00.732907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd25fa-5551-4b1d-ac7b-154d0c85c46b service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:00.743028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 697/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:03.191765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ac21-58c6-43fd-9ed3-2adb89333878 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:03.192608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ac21-58c6-43fd-9ed3-2adb89333878 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:03.192608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875ac21-58c6-43fd-9ed3-2adb89333878 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:03.194096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 695/1392] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:59:03.200231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8562b-c686-43e2-9cc4-0933372b4cdd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:03.200683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8562b-c686-43e2-9cc4-0933372b4cdd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:03.200822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8562b-c686-43e2-9cc4-0933372b4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:03.224979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fbb8562b-c686-43e2-9cc4-0933372b4cdd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:03.227178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fbb8562b-c686-43e2-9cc4-0933372b4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:03.237481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 698/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:03.283566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875ac21-58c6-43fd-9ed3-2adb89333878 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:03.298244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875ac21-58c6-43fd-9ed3-2adb89333878 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:03.303975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:03.304621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:03.304904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:03.305179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:03.314166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:03.315789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edafde6-9024-4113-9ab0-ca4b8f5c8c75 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:03.325582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 696/1394] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:59:05.518783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c69217-8ef2-4142-a1d1-aeda96de24f8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.519376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c69217-8ef2-4142-a1d1-aeda96de24f8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.519376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c69217-8ef2-4142-a1d1-aeda96de24f8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.520668 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 699/1395] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:05.521828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624ece-908a-4164-bab1-2687f29edefa None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.522261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624ece-908a-4164-bab1-2687f29edefa None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.522539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e624ece-908a-4164-bab1-2687f29edefa None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.524023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 697/1396] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:59:05.525903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.526186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.526521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.529535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd24cf-4e64-4525-b908-c2838cafd6c9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.529949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd24cf-4e64-4525-b908-c2838cafd6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.530391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd24cf-4e64-4525-b908-c2838cafd6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.546466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:05.547690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:05.549386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-63bd24cf-4e64-4525-b908-c2838cafd6c9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:05.550690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-63bd24cf-4e64-4525-b908-c2838cafd6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:05.556150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 700/1397] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:05.558793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 698/1398] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:05.600997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:05.613735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101d375-64c4-47eb-badb-f91733fdcc91 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:05.614925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448f28d-ab61-4be8-b69f-5dfa66c0115f None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.615220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448f28d-ab61-4be8-b69f-5dfa66c0115f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.615499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448f28d-ab61-4be8-b69f-5dfa66c0115f None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.616875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 699/1399] 10.222.0.49 () {54 vars in 962 bytes} [Thu Jun 30 12:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 12:59:05.619855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:05.620715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.621003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.621275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.621671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0993-9584-438c-b44f-448ada6f6987 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.621955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0993-9584-438c-b44f-448ada6f6987 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.622233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a0993-9584-438c-b44f-448ada6f6987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.630387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:05.631994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:05.641400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 701/1400] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:05.674067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d09a0993-9584-438c-b44f-448ada6f6987 None None] MFA Rules not processed for user `aaf234ea529642af8039f8e3b0bf601c`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:05.675345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d09a0993-9584-438c-b44f-448ada6f6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:05.741218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 700/1401] 10.222.0.49 () {58 vars in 1059 bytes} [Thu Jun 30 12:59:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:59:05.749347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:05.759187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a67c269-6d3b-4566-b55b-82589c28e026 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:05.763499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:05.764166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:05.764446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:05.764713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:05.775491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:05.777112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bfa24-2916-4fe1-acc5-55e64112b73d service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:05.788300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 702/1402] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:06.509740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4bd1d-dd0d-43eb-accb-dc54af41f24d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:06.510030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4bd1d-dd0d-43eb-accb-dc54af41f24d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:06.510307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a4bd1d-dd0d-43eb-accb-dc54af41f24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:06.536706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:59:06.560191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-71a4bd1d-dd0d-43eb-accb-dc54af41f24d None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:06.561411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-71a4bd1d-dd0d-43eb-accb-dc54af41f24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:06.626657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 701/1403] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 12:59:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:06.671035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7a21-328e-47c7-b64c-8433b6751c1e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:06.671500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7a21-328e-47c7-b64c-8433b6751c1e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:06.671621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7a21-328e-47c7-b64c-8433b6751c1e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:06.673061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 703/1404] 10.222.0.49 () {54 vars in 942 bytes} [Thu Jun 30 12:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:06.679314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05451b-9239-4ca7-9a64-29868ee1440a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:06.679626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05451b-9239-4ca7-9a64-29868ee1440a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:06.679902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05451b-9239-4ca7-9a64-29868ee1440a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:06.728505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5f05451b-9239-4ca7-9a64-29868ee1440a None None] MFA Rules not processed for user `2b2d6693e40d40f988254bbc7fcca45f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:06.729605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5f05451b-9239-4ca7-9a64-29868ee1440a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:06.805284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 702/1405] 10.222.0.49 () {58 vars in 1039 bytes} [Thu Jun 30 12:59:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:06.815036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7a21-328e-47c7-b64c-8433b6751c1e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:06.829450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7a21-328e-47c7-b64c-8433b6751c1e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:06.835243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:06.835881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:06.836162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:06.836432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:06.845685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:06.847321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa3769-6d1c-4d58-ae45-ccaed7717512 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:06.855888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 704/1406] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 12:59:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 12:59:07.712340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f82a-8b08-4dc9-85f9-4cc1e1e69c6e None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:07.712884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f82a-8b08-4dc9-85f9-4cc1e1e69c6e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:07.712884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52f82a-8b08-4dc9-85f9-4cc1e1e69c6e None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:07.714447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 703/1407] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 12:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:07.720683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3d8ad-baf7-4073-abf7-38957cb5eb21 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:07.720683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3d8ad-baf7-4073-abf7-38957cb5eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:07.720683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3d8ad-baf7-4073-abf7-38957cb5eb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:07.739368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aad3d8ad-baf7-4073-abf7-38957cb5eb21 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:07.740664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aad3d8ad-baf7-4073-abf7-38957cb5eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:07.754737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 705/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 12:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:07.794942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52f82a-8b08-4dc9-85f9-4cc1e1e69c6e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:07.811816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52f82a-8b08-4dc9-85f9-4cc1e1e69c6e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:07.817937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:07.818710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:07.818996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:07.819269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:07.828330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:07.829936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af9cf5-da2d-4dc2-885e-4e95c08f82d4 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:07.850528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 704/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:17.734524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=46 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:59:17.835276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=82 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:59:28.982982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdbd76-2924-4d12-a8c9-7003229e2e23 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:28.983564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdbd76-2924-4d12-a8c9-7003229e2e23 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:28.983564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdbd76-2924-4d12-a8c9-7003229e2e23 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:28.984961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 706/1410] 10.222.0.49 () {54 vars in 918 bytes} [Thu Jun 30 12:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:28.990511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14c86-deae-4592-a1a6-466f928c6bcc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:28.990999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14c86-deae-4592-a1a6-466f928c6bcc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:28.991242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14c86-deae-4592-a1a6-466f928c6bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:29.009913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-78d14c86-deae-4592-a1a6-466f928c6bcc None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:29.011150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-78d14c86-deae-4592-a1a6-466f928c6bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:29.021243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 705/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Jun 30 12:59:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:29.036924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280ef6-246f-4507-84e8-3cc371d33933 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:29.037216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280ef6-246f-4507-84e8-3cc371d33933 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:29.037496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280ef6-246f-4507-84e8-3cc371d33933 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:29.038897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 707/1412] 10.222.0.49 () {54 vars in 918 bytes} [Thu Jun 30 12:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:29.043578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3a190-41f3-4d61-adbd-9ba921c7f859 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:29.043865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3a190-41f3-4d61-adbd-9ba921c7f859 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:29.044137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3a190-41f3-4d61-adbd-9ba921c7f859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:29.063357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b5e3a190-41f3-4d61-adbd-9ba921c7f859 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:29.064538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b5e3a190-41f3-4d61-adbd-9ba921c7f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:29.079964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 706/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Thu Jun 30 12:59:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:31.574492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd01f2f-e106-496a-8d89-db46ab0a4fc3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:31.575453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd01f2f-e106-496a-8d89-db46ab0a4fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:31.575453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd01f2f-e106-496a-8d89-db46ab0a4fc3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:31.577441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:31.586593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabebc-0103-478e-b49d-170969fe25ae None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:31.587073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabebc-0103-478e-b49d-170969fe25ae None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:31.587751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcabebc-0103-478e-b49d-170969fe25ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:31.611191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7fcabebc-0103-478e-b49d-170969fe25ae None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:31.612384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7fcabebc-0103-478e-b49d-170969fe25ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:31.620775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:31.667408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd01f2f-e106-496a-8d89-db46ab0a4fc3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:31.683613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd01f2f-e106-496a-8d89-db46ab0a4fc3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:31.690046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:31.690721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:31.691001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:31.691266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:31.699975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:31.701571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5480dab7-55f7-48f3-aba9-7d2f85abfa39 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:31.711205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 709/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:32.604175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b936fd-6843-45b2-a155-4a61c00f8e7d None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:32.605038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b936fd-6843-45b2-a155-4a61c00f8e7d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:32.605038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b936fd-6843-45b2-a155-4a61c00f8e7d None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:32.607168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:32.617429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6094a-6c88-4542-9892-9f3077658da8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:32.617429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6094a-6c88-4542-9892-9f3077658da8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:32.617429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6094a-6c88-4542-9892-9f3077658da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:32.638140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-54d6094a-6c88-4542-9892-9f3077658da8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:32.639385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-54d6094a-6c88-4542-9892-9f3077658da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:32.649241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 12:59:32.800424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b936fd-6843-45b2-a155-4a61c00f8e7d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:32.817562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b936fd-6843-45b2-a155-4a61c00f8e7d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:32.824408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:32.825049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:32.825328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:32.825600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:32.834313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:32.835927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeff08c-229b-4467-a04b-1b48586502c1 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:32.845896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 709/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:33.725580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6796b-dec6-4ce3-85c4-4397112eed0f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:33.726186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6796b-dec6-4ce3-85c4-4397112eed0f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:33.726186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6796b-dec6-4ce3-85c4-4397112eed0f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:33.727539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:33.732634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d612-a0c2-477e-bf7d-32c743a9729b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:33.733085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d612-a0c2-477e-bf7d-32c743a9729b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:33.733200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208d612-a0c2-477e-bf7d-32c743a9729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:33.753016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7208d612-a0c2-477e-bf7d-32c743a9729b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:33.754184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7208d612-a0c2-477e-bf7d-32c743a9729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:33.764071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:33.917939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6796b-dec6-4ce3-85c4-4397112eed0f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:33.941558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6796b-dec6-4ce3-85c4-4397112eed0f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:33.949054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:33.949731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:33.950014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:33.950273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:33.959348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:33.960978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607278e-f815-4172-b0a0-9689365f4e27 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:33.971621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 712/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:34.846380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de49f-31d9-4817-9b3e-9b70e8e823a9 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:34.846956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de49f-31d9-4817-9b3e-9b70e8e823a9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:34.847063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de49f-31d9-4817-9b3e-9b70e8e823a9 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:34.849259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:34.857901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63dc7-c0ab-400a-a004-bfc9d380ccc5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:34.858307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63dc7-c0ab-400a-a004-bfc9d380ccc5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:34.858494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c63dc7-c0ab-400a-a004-bfc9d380ccc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:34.883218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-69c63dc7-c0ab-400a-a004-bfc9d380ccc5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:34.884415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-69c63dc7-c0ab-400a-a004-bfc9d380ccc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:34.893518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:35.047800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de49f-31d9-4817-9b3e-9b70e8e823a9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:35.072766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de49f-31d9-4817-9b3e-9b70e8e823a9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:35.080875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:35.081514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:35.081794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:35.082069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:35.091928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:35.093718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa8bdeb-47b2-443e-a32b-14b93f101f46 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:35.103982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 712/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 12:59:35.981534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e927e-3bb8-4f6a-965b-25bcb0e138a1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:35.982609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e927e-3bb8-4f6a-965b-25bcb0e138a1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:35.982609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e927e-3bb8-4f6a-965b-25bcb0e138a1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:35.984693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:35.989887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b368e-9be0-413b-a946-90e75a685421 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:35.990361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b368e-9be0-413b-a946-90e75a685421 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:35.990467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b368e-9be0-413b-a946-90e75a685421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:36.009580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-538b368e-9be0-413b-a946-90e75a685421 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:36.010810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-538b368e-9be0-413b-a946-90e75a685421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:36.019806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:36.167919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e927e-3bb8-4f6a-965b-25bcb0e138a1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:36.185059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e927e-3bb8-4f6a-965b-25bcb0e138a1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:36.191270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:36.191897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:36.192172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:36.192441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:36.200521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:36.202107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167233-c8c9-4ca3-b76a-1c1586b50319 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:36.211832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 715/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:37.013604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29376832-2d5a-4d91-bf83-96e8e6a8d531 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:37.014502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29376832-2d5a-4d91-bf83-96e8e6a8d531 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:37.014502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29376832-2d5a-4d91-bf83-96e8e6a8d531 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:37.016746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:37.022860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943ae60-e016-44d5-84bc-f5b696018bea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:37.023656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943ae60-e016-44d5-84bc-f5b696018bea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:37.023827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943ae60-e016-44d5-84bc-f5b696018bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:37.045876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a943ae60-e016-44d5-84bc-f5b696018bea None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:37.047202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a943ae60-e016-44d5-84bc-f5b696018bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:37.056156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:37.209921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29376832-2d5a-4d91-bf83-96e8e6a8d531 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:37.228883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29376832-2d5a-4d91-bf83-96e8e6a8d531 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:37.234959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:37.235745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:37.236015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:37.236287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:37.244989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:37.246591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693c707-a7e9-4db3-a1ef-5f92e2e4e33b service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:37.256283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 715/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:38.063111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c879-4215-4031-a3c3-f9658a1b779a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:38.063747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c879-4215-4031-a3c3-f9658a1b779a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:38.063747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c879-4215-4031-a3c3-f9658a1b779a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:38.065031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:38.071118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45ee6-2d99-49c4-875b-29a2d01e0803 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:38.071867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45ee6-2d99-49c4-875b-29a2d01e0803 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:38.072036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f45ee6-2d99-49c4-875b-29a2d01e0803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:38.093480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e6f45ee6-2d99-49c4-875b-29a2d01e0803 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:38.094742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e6f45ee6-2d99-49c4-875b-29a2d01e0803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:38.103157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:38.257417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285c879-4215-4031-a3c3-f9658a1b779a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:38.276221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285c879-4215-4031-a3c3-f9658a1b779a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:38.282593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:38.283260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:38.283548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:38.283824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:38.293007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:38.294630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c24b80-3341-4449-9c8e-34cab18c5a38 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:38.304644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 718/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:39.113006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6894c3-880e-4c92-b799-16aa3211fb38 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:39.113613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6894c3-880e-4c92-b799-16aa3211fb38 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:39.113613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6894c3-880e-4c92-b799-16aa3211fb38 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:39.115729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:39.124176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1998d-a45f-4d75-80bd-b2559d25a30f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:39.124568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1998d-a45f-4d75-80bd-b2559d25a30f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:39.125046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1998d-a45f-4d75-80bd-b2559d25a30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:39.145846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4fe1998d-a45f-4d75-80bd-b2559d25a30f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:39.147082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4fe1998d-a45f-4d75-80bd-b2559d25a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:39.155561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:39.305581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6894c3-880e-4c92-b799-16aa3211fb38 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:39.331207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6894c3-880e-4c92-b799-16aa3211fb38 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:39.342675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:39.344102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:39.350014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:39.350346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:39.365619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:39.368085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad5031-5415-4f11-9306-d958e6bf59bc service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:39.381112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 718/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:40.176644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9433-275f-4be7-b1b3-d4790d6fbd83 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:40.177212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9433-275f-4be7-b1b3-d4790d6fbd83 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:40.177212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c9433-275f-4be7-b1b3-d4790d6fbd83 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:40.179346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 12:59:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:40.185375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc659db-8260-4798-af4e-919ec3c4f09b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:40.186113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc659db-8260-4798-af4e-919ec3c4f09b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:40.186255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc659db-8260-4798-af4e-919ec3c4f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:40.207049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ecc659db-8260-4798-af4e-919ec3c4f09b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:40.208261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ecc659db-8260-4798-af4e-919ec3c4f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:40.219097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 12:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:40.381985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c9433-275f-4be7-b1b3-d4790d6fbd83 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:40.401081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c9433-275f-4be7-b1b3-d4790d6fbd83 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:40.407716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:40.408356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:40.408635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:40.408902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:40.417809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:40.419500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725ec6f-1895-4836-a9c7-b79c058be062 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:40.429379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 721/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:42.343499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780794d4-699d-4d6b-bd71-aa0c7e39a729 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:42.344120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780794d4-699d-4d6b-bd71-aa0c7e39a729 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:42.344120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780794d4-699d-4d6b-bd71-aa0c7e39a729 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:42.345515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:42.351083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8dd6-b002-4fe8-946b-b68e62990f7b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:42.351526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8dd6-b002-4fe8-946b-b68e62990f7b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:42.351641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8dd6-b002-4fe8-946b-b68e62990f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:42.370826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a18d8dd6-b002-4fe8-946b-b68e62990f7b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:42.372025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a18d8dd6-b002-4fe8-946b-b68e62990f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:42.381371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:42.419925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780794d4-699d-4d6b-bd71-aa0c7e39a729 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:42.439008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780794d4-699d-4d6b-bd71-aa0c7e39a729 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:42.443706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:42.444340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:42.444616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:42.444882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:42.454872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:42.456303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a896dd7-fe28-4c92-9a71-fdacad8333f3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:42.466639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 721/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:44.653589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dcc25-a389-420e-8e2b-9779ea321dfe None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:44.654202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dcc25-a389-420e-8e2b-9779ea321dfe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:44.654202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dcc25-a389-420e-8e2b-9779ea321dfe None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:44.655607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:44.662005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a15b-534b-4a3e-81d1-0669e8f16c71 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:44.662833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a15b-534b-4a3e-81d1-0669e8f16c71 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:44.663007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367a15b-534b-4a3e-81d1-0669e8f16c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:44.687456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e367a15b-534b-4a3e-81d1-0669e8f16c71 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:44.688715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e367a15b-534b-4a3e-81d1-0669e8f16c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:44.698342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:44.737551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dcc25-a389-420e-8e2b-9779ea321dfe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:44.751498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dcc25-a389-420e-8e2b-9779ea321dfe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:44.755481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:44.756111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:44.756386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:44.756652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:44.766555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:44.768028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ce600-928a-49ee-96e0-2ab161329fbb service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:44.777922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 724/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:47.105939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1998c-f747-424f-a47e-aedb30669837 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:47.106879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1998c-f747-424f-a47e-aedb30669837 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:47.106879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1998c-f747-424f-a47e-aedb30669837 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:47.109012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:47.114593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42c13-2b59-48ad-bb12-32503eb0f00b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:47.115038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42c13-2b59-48ad-bb12-32503eb0f00b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:47.115133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42c13-2b59-48ad-bb12-32503eb0f00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:47.134625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aca42c13-2b59-48ad-bb12-32503eb0f00b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:47.135810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aca42c13-2b59-48ad-bb12-32503eb0f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:47.144353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:47.185942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1998c-f747-424f-a47e-aedb30669837 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:47.203972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1998c-f747-424f-a47e-aedb30669837 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:47.208733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:47.209426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:47.209703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:47.209971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:47.220805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:47.222239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db93783-2f18-429c-bc2c-af1fc5ddb869 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:47.232100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 724/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:49.362900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc47646-e947-453c-98bd-b18a7d7de82d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:49.363795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc47646-e947-453c-98bd-b18a7d7de82d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:49.363795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc47646-e947-453c-98bd-b18a7d7de82d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:49.365997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:49.374810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb265b-6b08-4aa5-a769-6d7e6576928b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:49.375255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb265b-6b08-4aa5-a769-6d7e6576928b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:49.375378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb265b-6b08-4aa5-a769-6d7e6576928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:49.399009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ebeb265b-6b08-4aa5-a769-6d7e6576928b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:49.400221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ebeb265b-6b08-4aa5-a769-6d7e6576928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:49.408748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:49.450074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc47646-e947-453c-98bd-b18a7d7de82d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:49.462081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc47646-e947-453c-98bd-b18a7d7de82d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:49.466574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:49.467229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:49.467499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:49.467764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:49.477398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:49.478918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec39e-8286-449a-879e-ee602bda8025 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:49.488545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 727/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:51.643084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ca5-eb85-4fc1-9cad-266033c22cfd None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:51.643679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ca5-eb85-4fc1-9cad-266033c22cfd None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:51.643679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b84ca5-eb85-4fc1-9cad-266033c22cfd None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:51.645045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:51.651675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b01d8-320e-453c-ab73-016484f81bcf None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:51.652435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b01d8-320e-453c-ab73-016484f81bcf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:51.652631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b01d8-320e-453c-ab73-016484f81bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:51.678738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a05b01d8-320e-453c-ab73-016484f81bcf None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:51.679948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a05b01d8-320e-453c-ab73-016484f81bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:51.689803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:51.727715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84ca5-eb85-4fc1-9cad-266033c22cfd None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:51.739321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b84ca5-eb85-4fc1-9cad-266033c22cfd None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:51.742934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:51.743564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:51.743844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:51.744114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:51.753684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:51.755135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5273d-22d1-46d9-9a08-2e4797b8ec69 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:51.764527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 727/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:53.948569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2f5c-e12d-44cf-a593-ec72ee4ebc86 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:53.949085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2f5c-e12d-44cf-a593-ec72ee4ebc86 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:53.949085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf2f5c-e12d-44cf-a593-ec72ee4ebc86 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:53.950466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:53.957536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328d3e-2c82-4fd6-9d6a-2a299c9eacdc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:53.957985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328d3e-2c82-4fd6-9d6a-2a299c9eacdc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:53.958073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de328d3e-2c82-4fd6-9d6a-2a299c9eacdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:53.978621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-de328d3e-2c82-4fd6-9d6a-2a299c9eacdc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:53.980423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-de328d3e-2c82-4fd6-9d6a-2a299c9eacdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:53.990058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:54.032747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2f5c-e12d-44cf-a593-ec72ee4ebc86 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:54.050398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf2f5c-e12d-44cf-a593-ec72ee4ebc86 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:54.057074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:54.057914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:54.058239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:54.058630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:54.073445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:54.075465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614be31-ca8e-4806-bd11-df984fd86500 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:54.086584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 730/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:56.172102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a188f-aa3a-4594-9f97-821f492e37f2 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:56.172676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a188f-aa3a-4594-9f97-821f492e37f2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:56.172676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a188f-aa3a-4594-9f97-821f492e37f2 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:56.174033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:56.180586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef0ed-8241-4f93-b721-f0d51c0522e7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:56.181332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef0ed-8241-4f93-b721-f0d51c0522e7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:56.181480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef0ed-8241-4f93-b721-f0d51c0522e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:56.207707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-637ef0ed-8241-4f93-b721-f0d51c0522e7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:56.208898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-637ef0ed-8241-4f93-b721-f0d51c0522e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:56.218496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:56.259488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a188f-aa3a-4594-9f97-821f492e37f2 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:56.278939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a188f-aa3a-4594-9f97-821f492e37f2 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:56.285008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:56.285717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:56.285998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:56.286269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:56.297515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:56.298993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d34efe-eb67-4585-b569-a8c880592a23 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:56.308929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 730/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 12:59:58.373074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e84a4-e448-4952-8063-6977ce1988c7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:58.373805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e84a4-e448-4952-8063-6977ce1988c7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:58.373805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e84a4-e448-4952-8063-6977ce1988c7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:58.375216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 12:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 12:59:58.381368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6f6b-700d-43e6-a89c-7f5f42946c62 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:58.381368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6f6b-700d-43e6-a89c-7f5f42946c62 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:58.381796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c6f6b-700d-43e6-a89c-7f5f42946c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:58.390484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=79 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:59:58.403924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-024c6f6b-700d-43e6-a89c-7f5f42946c62 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 12:59:58.405285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-024c6f6b-700d-43e6-a89c-7f5f42946c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 12:59:58.413512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 12:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 12:59:58.454532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e84a4-e448-4952-8063-6977ce1988c7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 12:59:58.464243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=73 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 12:59:58.474103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e84a4-e448-4952-8063-6977ce1988c7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 12:59:58.480628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 12:59:58.481645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 12:59:58.482098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 12:59:58.482543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 12:59:58.492630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 12:59:58.494072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b4303-73cc-4198-84e8-c1f401511288 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 12:59:58.503901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 733/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 12:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:00.543490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5961b1-202d-4a92-bfa4-0a5e362eeba9 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:00.544293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5961b1-202d-4a92-bfa4-0a5e362eeba9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:00.544293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5961b1-202d-4a92-bfa4-0a5e362eeba9 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:00.546105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:00.551482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a397-f717-4a43-9aa1-89596a5346a9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:00.551932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a397-f717-4a43-9aa1-89596a5346a9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:00.552028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686a397-f717-4a43-9aa1-89596a5346a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:00.569975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3686a397-f717-4a43-9aa1-89596a5346a9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:00.571200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3686a397-f717-4a43-9aa1-89596a5346a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:00.579659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:00.620074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5961b1-202d-4a92-bfa4-0a5e362eeba9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:00.639977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5961b1-202d-4a92-bfa4-0a5e362eeba9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:00.644489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:00.645124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:00.645402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:00.645673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:00.655134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:00.656560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf943e47-36ff-46a0-887b-0636dd4f868a service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:00.665900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 733/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:02.709991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8078-ee89-4601-a1eb-3d2a616824bd None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:02.710942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8078-ee89-4601-a1eb-3d2a616824bd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:02.710942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf8078-ee89-4601-a1eb-3d2a616824bd None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:02.713165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:00:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:02.722140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be46c-b241-483e-b5f2-98549bd1655c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:02.722611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be46c-b241-483e-b5f2-98549bd1655c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:02.722762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02be46c-b241-483e-b5f2-98549bd1655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:02.746876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e02be46c-b241-483e-b5f2-98549bd1655c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:02.748236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e02be46c-b241-483e-b5f2-98549bd1655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:02.759334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:02.798527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf8078-ee89-4601-a1eb-3d2a616824bd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:02.811801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf8078-ee89-4601-a1eb-3d2a616824bd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:02.816196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:02.816834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:02.817111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:02.817379 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:02.827297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:02.828717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfac7c-559a-429a-8555-52cf78882632 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:02.838433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 736/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:12.739654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=11 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:00:12.834165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=14 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:00:22.669230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215379-2c44-42c4-bbbf-3261edc71ac6 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:22.669853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215379-2c44-42c4-bbbf-3261edc71ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:22.669853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7215379-2c44-42c4-bbbf-3261edc71ac6 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:22.671273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 735/1471] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:22.677815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004fa12-8ad8-44b4-ab8e-ed752d1e1b5b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:22.678560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004fa12-8ad8-44b4-ab8e-ed752d1e1b5b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:22.678774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004fa12-8ad8-44b4-ab8e-ed752d1e1b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:22.699726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a004fa12-8ad8-44b4-ab8e-ed752d1e1b5b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:22.701021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a004fa12-8ad8-44b4-ab8e-ed752d1e1b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:22.709844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 737/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:00:22.758148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daf70-a45e-4993-8ba3-cddeac20ce26 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:22.758474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daf70-a45e-4993-8ba3-cddeac20ce26 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:22.758724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5daf70-a45e-4993-8ba3-cddeac20ce26 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:22.760103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 736/1473] 10.222.0.49 () {54 vars in 962 bytes} [Thu Jun 30 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 13:00:22.765206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774cea-3ffb-4634-87a8-a957ea512554 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:22.765491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774cea-3ffb-4634-87a8-a957ea512554 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:22.765762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17774cea-3ffb-4634-87a8-a957ea512554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:22.785041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-17774cea-3ffb-4634-87a8-a957ea512554 None None] MFA Rules not processed for user `aaf234ea529642af8039f8e3b0bf601c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:22.786235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-17774cea-3ffb-4634-87a8-a957ea512554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:22.793922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 738/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Thu Jun 30 13:00:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:22.801716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5daf70-a45e-4993-8ba3-cddeac20ce26 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:22.819691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5daf70-a45e-4993-8ba3-cddeac20ce26 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:22.826205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:22.826895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:22.827173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:22.827445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:22.836956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:22.838400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0547525-406b-4c4b-978c-32e7159a0a47 service glance] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:22.848363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 737/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:24.970578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ce8cb-8c98-43aa-a2fa-79713ce1ad02 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:24.971256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ce8cb-8c98-43aa-a2fa-79713ce1ad02 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:24.971256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ce8cb-8c98-43aa-a2fa-79713ce1ad02 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:24.972714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 739/1476] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:24.979293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5c2a-f5d0-40da-8246-297b10cea05c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:24.979902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5c2a-f5d0-40da-8246-297b10cea05c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:24.980114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb5c2a-f5d0-40da-8246-297b10cea05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:24.999751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-66bb5c2a-f5d0-40da-8246-297b10cea05c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:25.000946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-66bb5c2a-f5d0-40da-8246-297b10cea05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:25.009724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 738/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:25.033217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ce8cb-8c98-43aa-a2fa-79713ce1ad02 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:25.046349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ce8cb-8c98-43aa-a2fa-79713ce1ad02 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:25.051315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:25.051954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:25.052231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:25.052498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:25.053217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:25.055032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:00:25.055032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:00:25.055032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:00:25.056175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a572345-c1b6-48d4-826e-a26607b3588d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:25.076258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 740/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Thu Jun 30 13:00:25 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:00:27.184045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf8a47-a118-46ee-98e4-afe4da488cc5 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:27.184616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf8a47-a118-46ee-98e4-afe4da488cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:27.184616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf8a47-a118-46ee-98e4-afe4da488cc5 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:27.186140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 739/1479] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:27.191843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d626cf-be0a-43db-b9cf-8f1aceb778f8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:27.192372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d626cf-be0a-43db-b9cf-8f1aceb778f8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:27.192372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d626cf-be0a-43db-b9cf-8f1aceb778f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:27.211268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-80d626cf-be0a-43db-b9cf-8f1aceb778f8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:27.212478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-80d626cf-be0a-43db-b9cf-8f1aceb778f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:27.221443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 741/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:27.268607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf8a47-a118-46ee-98e4-afe4da488cc5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:27.282848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf8a47-a118-46ee-98e4-afe4da488cc5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:27.288575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:27.289223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:27.289499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:27.289764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:27.298557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:27.300158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5276c6e-4fce-4f00-92bc-88cd978702aa service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:27.310133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 740/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:29.500178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b797b-d9c9-4c51-9dd1-9ca9662a76b5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:29.500807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b797b-d9c9-4c51-9dd1-9ca9662a76b5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:29.500807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b797b-d9c9-4c51-9dd1-9ca9662a76b5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:29.502068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 742/1482] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:29.507611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ee4c-f5a0-4fa9-892f-1f5e8da1fe92 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:29.508036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ee4c-f5a0-4fa9-892f-1f5e8da1fe92 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:29.508130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9ee4c-f5a0-4fa9-892f-1f5e8da1fe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:29.527999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-69c9ee4c-f5a0-4fa9-892f-1f5e8da1fe92 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:29.529131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-69c9ee4c-f5a0-4fa9-892f-1f5e8da1fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:29.538253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 741/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:29.575671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b797b-d9c9-4c51-9dd1-9ca9662a76b5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:29.588479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b797b-d9c9-4c51-9dd1-9ca9662a76b5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:29.595092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:29.596677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:29.597369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:29.598081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:29.613806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:29.615432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6a616-0054-4a5e-a21b-f448668367f8 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:29.627175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 743/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:31.696448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9b40a-1cbc-4380-a12a-55f93c3d3fbb None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:31.697050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9b40a-1cbc-4380-a12a-55f93c3d3fbb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:31.697050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9b40a-1cbc-4380-a12a-55f93c3d3fbb None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:31.698389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 742/1485] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:31.703931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742224-cce1-4072-8d67-df1472190075 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:31.704408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742224-cce1-4072-8d67-df1472190075 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:31.704504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4742224-cce1-4072-8d67-df1472190075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:31.725000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4742224-cce1-4072-8d67-df1472190075 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:31.726199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4742224-cce1-4072-8d67-df1472190075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:31.735114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 744/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:31.772623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9b40a-1cbc-4380-a12a-55f93c3d3fbb None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:31.785163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9b40a-1cbc-4380-a12a-55f93c3d3fbb None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:31.789406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:31.790038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:31.790337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:31.790624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:31.800090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:31.801511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbf988-c42e-451e-a0f3-7241fd5a4c8c service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:31.810871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 743/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:34.043824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d165-8f30-4d64-99e5-7eede1a52f2d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:34.044527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d165-8f30-4d64-99e5-7eede1a52f2d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:34.044527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869d165-8f30-4d64-99e5-7eede1a52f2d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:34.046113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 745/1488] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:34.052546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092a400-4d65-436f-8d0a-57ad852bdc00 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:34.053001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092a400-4d65-436f-8d0a-57ad852bdc00 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:34.053139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e092a400-4d65-436f-8d0a-57ad852bdc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:34.080795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e092a400-4d65-436f-8d0a-57ad852bdc00 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:34.081997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e092a400-4d65-436f-8d0a-57ad852bdc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:34.091440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 744/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:34.131464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869d165-8f30-4d64-99e5-7eede1a52f2d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:34.143630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869d165-8f30-4d64-99e5-7eede1a52f2d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:34.147548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:34.148176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:34.148448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:34.148717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:34.158812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:34.160243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd4d33-6675-4629-8a83-9855d2cee7c2 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:34.170424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 746/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:00:36.378713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e198c5-5272-491b-a2fd-d53b67d7dcc1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:36.379276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e198c5-5272-491b-a2fd-d53b67d7dcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:36.379276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e198c5-5272-491b-a2fd-d53b67d7dcc1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:36.381209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 745/1491] 10.222.0.49 () {54 vars in 926 bytes} [Thu Jun 30 13:00:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:00:36.389252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff47c-cf06-48a9-83ce-073e396d968f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:36.389675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff47c-cf06-48a9-83ce-073e396d968f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:36.389805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff47c-cf06-48a9-83ce-073e396d968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:36.409917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5ecff47c-cf06-48a9-83ce-073e396d968f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:00:36.411144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ecff47c-cf06-48a9-83ce-073e396d968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:00:36.419839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 747/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Thu Jun 30 13:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:00:36.462666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e198c5-5272-491b-a2fd-d53b67d7dcc1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:00:36.482756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e198c5-5272-491b-a2fd-d53b67d7dcc1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:00:36.487130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:00:36.487770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:00:36.488048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:00:36.488320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:00:36.498255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:00:36.499720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db075d-c000-4ca4-8c32-2505e77a67dd service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:00:36.509908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 746/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:00:46.404272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=33 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:00:46.505146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=33 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:02:02.810460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:02.811101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:02.811101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:02.831175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:02:02.832373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:02.841883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 748/1494] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:02:02.850002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:02.850534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:02.850534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:02.869169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:02:02.870379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:02.878786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 747/1495] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:02:02.883343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:02.894052 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08aa09-b439-4b40-92a5-0e1e7ca9864b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:02.902528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:02.903571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:02.903909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:02.904232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:02.905361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:02:02.905814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:02.907446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:02.907446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:02:02.907446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:02:02.908729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:02.924053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 749/1496] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:02:02.933078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:02.946681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e781fe-68a4-4e43-af8d-effc4bf03a0b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:02.952488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:02.953135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:02.953422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:02.953686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:02.954376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:02.956189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:02.956189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:02:02.956189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:02:02.957390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.005189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 748/1497] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:02:02 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:02:03.010313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.021704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d36d2-bcda-48b1-aa8e-4131d9669f3a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.090782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.091331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.091599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.091874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.092556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.093587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:03.093587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:02:03.093587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:02:03.094839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.118582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 750/1498] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:02:03 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:02:03.123285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.133719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991d490-6728-4664-8bd7-0423fd630d6b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.139418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.140078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.140358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.140632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.141364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.142412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:03.142412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:02:03.142412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:02:03.143302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.156850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 749/1499] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:02:03.161057 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.171421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33201f-c5c1-4568-a830-d88bbb22c2f8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.176473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.177124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.177400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.177672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.178458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.179470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:03.179470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:02:03.179470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:02:03.180347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae0ab9-914f-4440-8d30-5f371b79ca7d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.193240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 751/1500] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:02:03.197235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.212218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f1b68-a2c6-40d6-9502-0357d16121bb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.219516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.220403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.220685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.220957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] PATH_INFO: `/v3/projects/078f2d34c41d4e1fbc70698553d48c08/users/9d548f9caae14eea9649649d135d1710/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.237311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=078f2d34c41d4e1fbc70698553d48c08, user_id=9d548f9caae14eea9649649d135d1710, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.238353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:03.238353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:02:03.238353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:02:03.240309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b772ddfd-289f-49cb-8c2f-c5d27874fba5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.255708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 750/1501] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:02:03 2022] PUT /identity/v3/projects/078f2d34c41d4e1fbc70698553d48c08/users/9d548f9caae14eea9649649d135d1710/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:02:03.268078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265b65-e985-4295-8694-3da097bdbdd0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.268365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265b65-e985-4295-8694-3da097bdbdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.268636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08265b65-e985-4295-8694-3da097bdbdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.296097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-08265b65-e985-4295-8694-3da097bdbdd0 None None] MFA Rules not processed for user `9d548f9caae14eea9649649d135d1710`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:02:03.297296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-08265b65-e985-4295-8694-3da097bdbdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:03.368753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 752/1502] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:02:03.384689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1ae5-d252-42cc-8692-6fc1738ee4b9 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.384977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1ae5-d252-42cc-8692-6fc1738ee4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.385282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1ae5-d252-42cc-8692-6fc1738ee4b9 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.386756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 751/1503] 10.222.0.49 () {54 vars in 963 bytes} [Thu Jun 30 13:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:02:03.393450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d483-59c0-471a-825c-5fa729e9c6a1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.393754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d483-59c0-471a-825c-5fa729e9c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.394014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6490d483-59c0-471a-825c-5fa729e9c6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.413419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6490d483-59c0-471a-825c-5fa729e9c6a1 None None] MFA Rules not processed for user `607bae995849489db891582db395e28c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:02:03.414645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6490d483-59c0-471a-825c-5fa729e9c6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:03.459410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:02:03.468449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 753/1504] 10.222.0.49 () {58 vars in 1060 bytes} [Thu Jun 30 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:02:03.480094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1ae5-d252-42cc-8692-6fc1738ee4b9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.491340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1ae5-d252-42cc-8692-6fc1738ee4b9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.495474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.496113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.496392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.496659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.505084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.506541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.515321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 752/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:02:03.559087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d115-82e3-47d5-884b-b2212188474e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.559378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d115-82e3-47d5-884b-b2212188474e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.559655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d115-82e3-47d5-884b-b2212188474e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.560953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 754/1506] 10.222.0.49 () {52 vars in 829 bytes} [Thu Jun 30 13:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:02:03.564633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.626656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564468f-fed2-4157-a269-3dfd5076eebb service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.631472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.632345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.632620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.632892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] PATH_INFO: `/v3/users/9d548f9caae14eea9649649d135d1710` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.634109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d548f9caae14eea9649649d135d1710)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.635231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:03.635231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:02:03.635231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:02:03.636530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.712010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d548f9caae14eea9649649d135d1710'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:03.724686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d548f9caae14eea9649649d135d1710'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:03.725437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d548f9caae14eea9649649d135d1710'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:03.746081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d548f9caae14eea9649649d135d1710'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:03.747245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d548f9caae14eea9649649d135d1710'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:03.757387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:02:03.773656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 753/1507] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:02:03 2022] DELETE /identity/v3/users/9d548f9caae14eea9649649d135d1710 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:02:03.778482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.778773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.779089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.796973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:02:03.798177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:03.860259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 755/1508] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:02:03.871107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:03.872645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:03.935217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46426687-66fe-4be9-a599-20e072d485fc admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:03.939244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:03.939890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:03.940174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:03.940448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:03.950213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:03.952148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a898a1-5344-45eb-9a50-b709265b378b service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:03.961837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 754/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:02:04.293063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:02:04.294592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:02:04.307086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adeda10-467f-410d-ac7c-2b1179f007db None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:02:04.313239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:02:04.314214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:02:04.314383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:02:04.314657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] PATH_INFO: `/v3/projects/078f2d34c41d4e1fbc70698553d48c08` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:02:04.316554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=078f2d34c41d4e1fbc70698553d48c08)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:02:04.317627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:02:04.317627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:02:04.317627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:02:04.318860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:02:04.383268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078f2d34c41d4e1fbc70698553d48c08', 'request_id': 'req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:04.396420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078f2d34c41d4e1fbc70698553d48c08', 'request_id': 'req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:04.410846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078f2d34c41d4e1fbc70698553d48c08', 'request_id': 'req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:04.422370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] The token cache is being invalidate because project 078f2d34c41d4e1fbc70698553d48c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:02:04.422722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:04.423523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1acadf3-aae4-4d4b-a8fd-e4ca8a9e0bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:02:04.425107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 756/1510] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:02:04 2022] DELETE /identity/v3/projects/078f2d34c41d4e1fbc70698553d48c08 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:02:13.957320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=29 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:02:14.419249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:03:42.168705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:42.169709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:42.169709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:42.171868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 13:03:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:03:42.177832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8184-d694-4fd3-b07c-cea0f2d717a2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:42.178632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8184-d694-4fd3-b07c-cea0f2d717a2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:42.178805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab8184-d694-4fd3-b07c-cea0f2d717a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:42.202516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-66ab8184-d694-4fd3-b07c-cea0f2d717a2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:42.203740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66ab8184-d694-4fd3-b07c-cea0f2d717a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:42.268665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 13:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:03:42.424700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:03:42.427594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:42.518059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46094f1c-e5ca-44cb-96eb-26aa915754cf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:03:42.523992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:03:42.524634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:42.524933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:42.525172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:42.534199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:03:42.535855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b55ad-46f1-4b34-bc3a-34727a40fa81 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:03:42.545781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 756/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:03:43.378069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791b992-efec-4e8c-a652-e7b06e1f351c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:43.378680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791b992-efec-4e8c-a652-e7b06e1f351c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:43.378680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791b992-efec-4e8c-a652-e7b06e1f351c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:43.380751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 13:03:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:03:43.388213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721188-0423-43aa-a1bc-c49d1e1481a7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:43.388691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721188-0423-43aa-a1bc-c49d1e1481a7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:43.388793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14721188-0423-43aa-a1bc-c49d1e1481a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:43.411113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-14721188-0423-43aa-a1bc-c49d1e1481a7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:43.412330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-14721188-0423-43aa-a1bc-c49d1e1481a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:43.420100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:03:43.572821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791b992-efec-4e8c-a652-e7b06e1f351c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:03:43.589900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791b992-efec-4e8c-a652-e7b06e1f351c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:03:43.597915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:03:43.598591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:43.598873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:43.599146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:43.608274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:03:43.609902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08435c15-efe9-4208-8fc2-73852daa6a4d service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:03:43.619700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 759/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:03:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:03:44.399869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449bf0-8104-4c4d-bbe8-24276f11b1f3 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:44.400467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449bf0-8104-4c4d-bbe8-24276f11b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:44.400467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af449bf0-8104-4c4d-bbe8-24276f11b1f3 None None] PATH_INFO: `/v3` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:44.402834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 758/1517] 10.222.0.26 () {54 vars in 925 bytes} [Thu Jun 30 13:03:44 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:03:44.410111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b92e4-c659-44a3-92d9-91fd5dbfac3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:44.410588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b92e4-c659-44a3-92d9-91fd5dbfac3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:44.410720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b92e4-c659-44a3-92d9-91fd5dbfac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:44.429999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4c4b92e4-c659-44a3-92d9-91fd5dbfac3b None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:44.431238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4c4b92e4-c659-44a3-92d9-91fd5dbfac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:44.502040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 760/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 30 13:03:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:03:53.402535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=37 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:03:53.991909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4a13-bc36-44f2-8940-81f3c438faec None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:53.992133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4a13-bc36-44f2-8940-81f3c438faec None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:53.992332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa4a13-bc36-44f2-8940-81f3c438faec None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:53.993740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 759/1519] 10.222.0.49 () {54 vars in 960 bytes} [Thu Jun 30 13:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:03:53.999023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c9e54-1501-4ca8-9f52-165ee9cf8660 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:53.999528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c9e54-1501-4ca8-9f52-165ee9cf8660 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:53.999528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c9e54-1501-4ca8-9f52-165ee9cf8660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:54.018368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b43c9e54-1501-4ca8-9f52-165ee9cf8660 None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:54.019563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b43c9e54-1501-4ca8-9f52-165ee9cf8660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:54.030078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 761/1520] 10.222.0.49 () {58 vars in 1057 bytes} [Thu Jun 30 13:03:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:03:54.038049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4a13-bc36-44f2-8940-81f3c438faec None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:03:54.053003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa4a13-bc36-44f2-8940-81f3c438faec None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:03:54.058252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:03:54.058946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:54.059222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:54.059499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:54.074694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:03:54.076269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274686c-37b2-4257-ad44-3ebee167d6eb service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:03:54.086687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 760/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:03:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:03:54.892201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408186e8-e8cd-4fd4-9c6b-7f93ce424681 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:54.892601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408186e8-e8cd-4fd4-9c6b-7f93ce424681 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:54.892936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408186e8-e8cd-4fd4-9c6b-7f93ce424681 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:54.895288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 13:03:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:03:54.900874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681e18c-ea69-4dad-8710-f808b2e46660 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:54.901401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681e18c-ea69-4dad-8710-f808b2e46660 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:54.901401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681e18c-ea69-4dad-8710-f808b2e46660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:54.920935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7681e18c-ea69-4dad-8710-f808b2e46660 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:54.922146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7681e18c-ea69-4dad-8710-f808b2e46660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:54.931189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 13:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:03:55.083270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408186e8-e8cd-4fd4-9c6b-7f93ce424681 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:03:55.098080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408186e8-e8cd-4fd4-9c6b-7f93ce424681 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:03:55.103700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:03:55.104329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:55.104603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:55.104871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:55.113369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:03:55.115303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469920e1-a141-4d28-8e42-b75741157a41 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:03:55.124761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 763/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:03:55.969649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559892-0a7c-4f9c-b397-e6c4d528732f None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:55.970541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559892-0a7c-4f9c-b397-e6c4d528732f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:55.970541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00559892-0a7c-4f9c-b397-e6c4d528732f None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:55.972770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 13:03:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:03:55.978789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05a88-25bd-4064-8326-87192343b425 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:55.979304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05a88-25bd-4064-8326-87192343b425 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:55.979781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a05a88-25bd-4064-8326-87192343b425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:56.002469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-15a05a88-25bd-4064-8326-87192343b425 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:03:56.003681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15a05a88-25bd-4064-8326-87192343b425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:03:56.013032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 13:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:03:56.182313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559892-0a7c-4f9c-b397-e6c4d528732f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:03:56.198278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00559892-0a7c-4f9c-b397-e6c4d528732f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:03:56.204365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:03:56.205002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:03:56.205284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:03:56.205557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:03:56.215045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:03:56.216656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039720f-ca75-4be5-a85f-af388bfa2197 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:03:56.226943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 763/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:04:00.633009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:00.633570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:00.633570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:00.635010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:04:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:04:00.641029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc493af-d695-4c2c-a2ff-5292295163fb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:00.641509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc493af-d695-4c2c-a2ff-5292295163fb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:00.641613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc493af-d695-4c2c-a2ff-5292295163fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:00.661614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5fc493af-d695-4c2c-a2ff-5292295163fb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:04:00.663100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5fc493af-d695-4c2c-a2ff-5292295163fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:00.671856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:04:00.710128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:04:00.712066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:00.772312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b9dc4-0fd9-4815-8c8c-a1bbb032b3db None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:04:00.779016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:04:00.780017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:00.780419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:00.780882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:00.793812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:04:00.795222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf047cfb-5efd-48fd-80d9-bcd9a5a4a163 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:04:00.800344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:04:00.805512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 766/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:04:02.813379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b82025-f42d-4616-9c4c-5facc216f3ba None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:02.814275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b82025-f42d-4616-9c4c-5facc216f3ba None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:02.814275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b82025-f42d-4616-9c4c-5facc216f3ba None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:02.816413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:04:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:04:02.824906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163c1c0-4fa1-400d-bd06-c9274ec45c56 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:02.825583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163c1c0-4fa1-400d-bd06-c9274ec45c56 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:02.825840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163c1c0-4fa1-400d-bd06-c9274ec45c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:02.848655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5163c1c0-4fa1-400d-bd06-c9274ec45c56 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:04:02.849934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5163c1c0-4fa1-400d-bd06-c9274ec45c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:02.859086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:04:02.897403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b82025-f42d-4616-9c4c-5facc216f3ba None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:04:02.910782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b82025-f42d-4616-9c4c-5facc216f3ba None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:04:02.915760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:04:02.916420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:02.916694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:02.916959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:02.926862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:04:02.928460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72bbe9-bc53-4b4b-a49d-29f1a958072b service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:04:02.938345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 766/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:04:06.541112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:06.541802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:06.541947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:06.544297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:04:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:04:06.552920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79fe2-b509-49c5-9c67-01035f2cd555 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:06.553346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79fe2-b509-49c5-9c67-01035f2cd555 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:06.553449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a79fe2-b509-49c5-9c67-01035f2cd555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:06.572951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-43a79fe2-b509-49c5-9c67-01035f2cd555 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:04:06.574154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-43a79fe2-b509-49c5-9c67-01035f2cd555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:06.582661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:04:06.886140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:04:06.887939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:06.949161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7210af-eb51-456d-99e0-362d02508cc1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:04:06.953135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:04:06.953775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:06.954048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:06.954345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:06.963951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:04:06.965716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79173c-bbdc-49db-ba9b-106792d67545 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:04:06.975403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 769/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:04:09.019035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:09.019634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:09.019634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:09.021004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 13:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:04:09.029529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0269d-4913-45fb-8453-a6eb4f741eec None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:09.030198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0269d-4913-45fb-8453-a6eb4f741eec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:09.030511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0269d-4913-45fb-8453-a6eb4f741eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:09.056649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-caf0269d-4913-45fb-8453-a6eb4f741eec None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:04:09.058053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-caf0269d-4913-45fb-8453-a6eb4f741eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:09.066065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:04:09.366747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:04:09.368820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:04:09.384057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58ed98-3fd6-4201-8837-2c129ac23af4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:04:09.388363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:04:09.388997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:04:09.389277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:04:09.389550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:04:09.399459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:04:09.401171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87af7-4845-4245-bff6-60891499b2d8 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:04:09.414268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 769/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:04:19.050821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=32 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:04:19.406981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=33 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:39.172718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.173466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.173466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.195269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.196492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.209732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 771/1540] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.219087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.220049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.220567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.246951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.248222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.259903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 770/1541] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.264880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.285275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8301d-3704-477b-ae78-cd9f82a511a2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.294352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.295065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.295349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.295626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.296375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.296686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.298337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f302-800a-40fe-9d72-5060bce711a6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.319844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 772/1542] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.327908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.341180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201605cc-b77a-4cb7-99b7-fd3a798b5d17 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.355033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.355721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.356001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.356278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.357228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.359354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b91217-8fb9-4089-9536-82fee1a50925 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.382128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.382524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.382796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.402769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 771/1543] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:39.404335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c301-2e24-4e50-b2a6-1200311467e6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.404624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c301-2e24-4e50-b2a6-1200311467e6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.404903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4c301-2e24-4e50-b2a6-1200311467e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.418568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.419800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.435568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-86b4c301-2e24-4e50-b2a6-1200311467e6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.436726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-86b4c301-2e24-4e50-b2a6-1200311467e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.517803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 773/1544] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:05:39.518758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.527441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 772/1545] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.529212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0198-573e-40cd-ba18-1a0fd7bc2619 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.529536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fb263-13e9-47a1-a52a-b6daf89e215c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.529727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0198-573e-40cd-ba18-1a0fd7bc2619 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.530252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c0198-573e-40cd-ba18-1a0fd7bc2619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.540072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.541012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.541474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.541748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.542495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.548673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8baddb-b9a5-492f-83a3-e35642024556 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.554160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-369c0198-573e-40cd-ba18-1a0fd7bc2619 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.555435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-369c0198-573e-40cd-ba18-1a0fd7bc2619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.563687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 773/1546] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.566455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c9b58-84e4-48cc-97f8-f25147cde450 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.566455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c9b58-84e4-48cc-97f8-f25147cde450 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.566455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c9b58-84e4-48cc-97f8-f25147cde450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.572000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 774/1547] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/users => generated 289 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:39.573426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67473-cd88-4e27-93a1-59c8ed466b17 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.574593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67473-cd88-4e27-93a1-59c8ed466b17 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.575030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a67473-cd88-4e27-93a1-59c8ed466b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.585914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-544c9b58-84e4-48cc-97f8-f25147cde450 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.587160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-544c9b58-84e4-48cc-97f8-f25147cde450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.593614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a1a67473-cd88-4e27-93a1-59c8ed466b17 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.594825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1a67473-cd88-4e27-93a1-59c8ed466b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.595671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 774/1548] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.596890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881de68-9a29-4b0f-9163-996fa8a03acc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.597177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881de68-9a29-4b0f-9163-996fa8a03acc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.597453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881de68-9a29-4b0f-9163-996fa8a03acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.603809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 775/1549] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.605066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9169-147e-4b86-bd14-14a15755a0c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.605352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9169-147e-4b86-bd14-14a15755a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.605626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f9169-147e-4b86-bd14-14a15755a0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.623412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c881de68-9a29-4b0f-9163-996fa8a03acc None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.624069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-021f9169-147e-4b86-bd14-14a15755a0c5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.624630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c881de68-9a29-4b0f-9163-996fa8a03acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.625235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-021f9169-147e-4b86-bd14-14a15755a0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.638344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 776/1550] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.638521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 775/1551] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.638622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.638622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.638808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b811ee-1d10-436a-b4ac-feff83d19bd2 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.638903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.639003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b811ee-1d10-436a-b4ac-feff83d19bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.639003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b811ee-1d10-436a-b4ac-feff83d19bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.654038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e9b811ee-1d10-436a-b4ac-feff83d19bd2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.655286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e9b811ee-1d10-436a-b4ac-feff83d19bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.656724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.657901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.663809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 776/1552] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.665103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f813-8e80-452f-bbce-6c89bc779a9d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.665395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f813-8e80-452f-bbce-6c89bc779a9d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.665679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5f813-8e80-452f-bbce-6c89bc779a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.666282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 777/1553] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.667074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.676991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d7100a-341d-421a-8b8e-59714409e656 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.681647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.682365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.682651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.682927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.683662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.685365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d282f36-9e7c-41de-9c0a-8c1125d9440f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.688623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7ec5f813-8e80-452f-bbce-6c89bc779a9d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.689777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7ec5f813-8e80-452f-bbce-6c89bc779a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.695375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 778/1554] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:39.696682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c129-acea-4afd-8d35-bcdab38b5aa9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.696970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c129-acea-4afd-8d35-bcdab38b5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.697245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c129-acea-4afd-8d35-bcdab38b5aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.698627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 777/1555] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.699929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.700223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.700503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.715610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1db1c129-acea-4afd-8d35-bcdab38b5aa9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.716815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1db1c129-acea-4afd-8d35-bcdab38b5aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.719150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.720335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.724966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 779/1556] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.725985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.726267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.726567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.732083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 778/1557] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.733010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.743002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4742b7-5927-4794-be8b-b3e620b99a08 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.747518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.748606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.748763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.749292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.749571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.749843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.750611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.750922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.751937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:39.751937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:05:39.751937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:05:39.752868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de57f371-96bb-404f-9f0d-44ede75fd535 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.757854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 780/1558] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.758673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.768891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 779/1559] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.768891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.768891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.768891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.769311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3731c-5407-43f6-af44-8fd63d0eef08 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.774018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.774737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.775042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.775309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.776041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.776340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.777950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.786260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:39.787582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:39.790351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 781/1560] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.791239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.796143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 780/1561] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:39.796920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.803699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a01c87-137e-44c6-a16a-176c81a0693c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.806683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2866e64-8124-4010-8c7c-5f4ac224f8a8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.808335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.808987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.809267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.809538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.810272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.811282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.811883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.811999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.812193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.812468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.813189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.813525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.815150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.826583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 782/1562] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:39.826583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.834072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 781/1563] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.834072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.837238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bccfa-ea74-4927-8f62-b9bd9429e338 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.837681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62069c-bc2b-4bf0-a708-a3b9209f7684 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.842624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.842917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.843269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.843551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.843710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.843807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.843906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.844229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.844619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.844929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.845046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.845266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.846548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.846888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.859438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 782/1564] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.859611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 783/1565] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.860583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.865377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.870942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbe19b-4df6-42ab-9e33-821e4206e8a8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.876110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.876675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0eb64-4f45-4dea-9ee6-eec693416bbd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.876793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.877034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.877311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.878036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:39.878371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.879981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.881469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.881979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.882261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.882615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.883287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.884311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:39.884311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:05:39.884311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:05:39.885484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.892679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 783/1566] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:39.893552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.906594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba07c5-0862-4e17-b469-63aa6572c9a9 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:39.914395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:39.915176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:39.915524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:39.915809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:39.916513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:39.916696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 784/1567] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:39.917609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.918761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:39.939428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:39.947267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 784/1568] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:39.948064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:39.962677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:39.986544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c4e6d-fd09-4d24-a661-d3093c7c9151 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.012875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4e813-15ba-47f6-9f5d-36df4c04ffa4 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.017372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.018042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.018313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.018615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.019311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.021276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.043430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.044101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.044411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.044686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.045568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:40.045881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.047527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.052111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 785/1569] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.052111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.060317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 785/1570] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:40.061390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.078351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964ae80-1658-4749-a8c7-7192c12e00df admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.113509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d25044-af06-4e72-a247-96c4c10fd647 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.118355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.119249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.119529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.119813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] PATH_INFO: `/v3/projects/988a6aa0bbb045bb9b66810219206ebc/users/2e84715279114df09ecfe4335c926066/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.134161 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=988a6aa0bbb045bb9b66810219206ebc, user_id=2e84715279114df09ecfe4335c926066, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.135757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.136425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.136702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.136892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.137036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.137685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.139689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.151235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 786/1571] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:40 2022] PUT /identity/v3/projects/988a6aa0bbb045bb9b66810219206ebc/users/2e84715279114df09ecfe4335c926066/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:40.152025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.172474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 786/1572] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.172474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.214720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4379-2a95-4dfc-ad87-02c6b3f150ac admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.220957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:40.236453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41b1f-2859-4c45-be58-fe13ec7abff8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.239444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:40.241263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.241921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.242197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.242535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.243197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.245128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.274426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 787/1573] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.275370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.281235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.281909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.282191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.282496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.283200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.285169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.316470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 787/1574] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.316470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.341141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220cab-695d-4ff9-8dec-c9bcc5a2a95e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.375490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd763ed3-7149-4d27-9787-cd2852218742 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.380633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.381304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.381584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.381820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.382577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.383606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:40.383606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:05:40.383606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:05:40.384815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.397631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.398305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.398613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.398881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.399595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.401518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.407557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 788/1575] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.407557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.418846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef3bda-d0b3-49f2-95f9-a233354feafa admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.423430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.423869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 788/1576] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.424084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.424353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.424634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.425352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.427815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ef682-0e7b-4287-a921-349c1d1c3912 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.427944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.437125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e4cca-df32-47c8-8ee3-efac66bde98d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.441809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.442532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.442774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.443015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.443759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.445588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.452033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 789/1577] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.452033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.452033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.452033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.469375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:40.470665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:40.477088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 789/1578] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.477711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.478949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 790/1579] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:40.479759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.489708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2fac3-f03a-43f6-97f1-da0ead28b722 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.530630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:40.546697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4954e74-08d4-4d7e-bf43-3b09a49ad5b2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.546882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.547460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.547742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.548019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.548714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.550744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.551367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.552031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.552339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.552611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.553296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.555226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.574312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 791/1580] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.574312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.578103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 790/1581] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.579036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.582382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703a6fc-c7fd-41bd-a57b-92ef6965133b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.588029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.589588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.589588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.589588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.589809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3fc59-0b9e-4a7b-a014-a06927e33490 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.590270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.592291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.595172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.595835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.596110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.596382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.597234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.598858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.599579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:40.610154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 791/1582] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.614473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.616719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 792/1583] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.619291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.624459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760c17b-4800-46de-93ae-993ff614dead admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.629417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.630072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.630194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcdecb-d9d8-4dd2-a8c7-fb62d20a8e6b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.630382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.630659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.631387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.633114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.634856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.635502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.635778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.636044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.636784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.638470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.643753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 792/1584] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.644624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.655949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 793/1585] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.655949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.656231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd97271-63a4-4e31-81e6-c3680495e373 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.658973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.659654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.659936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.660208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.660935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.662935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.688047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 793/1586] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:40 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:40.688047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.695663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272724-6336-4d3c-945b-4ae190078bf8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.700479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.701134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.701412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.701682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.702430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.704163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.708563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1bfd-486b-4d44-a2b0-47cef1a23f5b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.712227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.712941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.713292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.713626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.715055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 794/1587] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.715931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.723632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.725004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.725197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbea645-3c1d-4f86-98d5-05818303decf admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.729656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.730282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.730592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.730862 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.731570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.733137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.734488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 794/1588] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:40.735273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.744467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca98df5-65e0-4f18-9628-445aad5348c7 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.744692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 795/1589] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.745704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.751796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.752685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.752955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.753223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.753945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.755677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.756149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69b3ac-6b55-4190-802c-3e82407e8e54 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.761702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.762449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.762729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.763000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.763734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.765469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.767358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 795/1590] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.768263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.779904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 796/1591] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.780143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813a909-1b70-4296-869d-195dded55420 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.780377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.785445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.786478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.787330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.787609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.788345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.790296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.790815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b68c4c-8fcd-488e-8082-7ea372396fb6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.796207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.796948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.797288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.798207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.798838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.804159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.804313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 796/1592] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.804313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.813089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21904add-c704-4cec-954d-89f4e0fe99be admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.814935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 797/1593] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.815848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.821193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.821937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.822283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.822667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.823450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.825078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.830084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c606149-e694-4b76-a8b4-91b479604a47 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.837096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.837973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.838443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.838924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 797/1594] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.840065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.842609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.843666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.845571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.852924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc66fdb-5019-48d3-b314-b8ceefd03d47 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.860428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 798/1595] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.860428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.860694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.861350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.861630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.861901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.862663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.864485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.869421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbda657-1fff-44d0-a1b1-8d40d0898d57 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.874509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.875436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.875815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.875939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 798/1596] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.876035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] PATH_INFO: `/v3/projects/fb73f41e81224744800e167e0eb69863/users/126e2b6fc7114ec3bdc1bb3a99fcf752/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.876818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.887141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b061055-1f96-4804-8319-0d8c43e5097b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.891786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb73f41e81224744800e167e0eb69863, user_id=126e2b6fc7114ec3bdc1bb3a99fcf752, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.892333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.892841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:40.892841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:05:40.892841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:05:40.893236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.893577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.893911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.894729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:40.894873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.896474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:40.907421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 799/1597] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:40.908554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.910935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 799/1598] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:40 2022] PUT /identity/v3/projects/fb73f41e81224744800e167e0eb69863/users/126e2b6fc7114ec3bdc1bb3a99fcf752/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:40.911575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:40.924037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b788a-f03c-4cc0-b7f2-6269b81ec7e3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.991207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f021f-61b2-4936-9a54-fae4195cccf0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:40.997016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.997908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.998162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:40.998341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.998479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] PATH_INFO: `/v3/projects/2671e576de2c44e6b1f5df7c53aae76a/users/e0df4b789a804a00a267bcc5cf778324/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:40.999116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:40.999399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:40.999679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] PATH_INFO: `/v3/projects/22ec4cf465ec4c01be414c7f55351b20/users/8835c23d3ec04b9fa91442d34ce0f84e/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.016235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2671e576de2c44e6b1f5df7c53aae76a, user_id=e0df4b789a804a00a267bcc5cf778324, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.017776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ec4cf465ec4c01be414c7f55351b20, user_id=8835c23d3ec04b9fa91442d34ce0f84e, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.019385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.020803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.031165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.038042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 800/1599] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:40 2022] PUT /identity/v3/projects/22ec4cf465ec4c01be414c7f55351b20/users/8835c23d3ec04b9fa91442d34ce0f84e/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.038310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 800/1600] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:40 2022] PUT /identity/v3/projects/2671e576de2c44e6b1f5df7c53aae76a/users/e0df4b789a804a00a267bcc5cf778324/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.039163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.039509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.052026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.127389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5634448f-9542-49b0-a34c-a307e9e3f739 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.133169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.133976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.135083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.135462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.136306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.138010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.147803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b22c54-1c32-4895-95b0-01c0901cb83c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.156749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 801/1601] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:41.156749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.165349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.166446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.166832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.167205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] PATH_INFO: `/v3/projects/1737469298f94e1b86b7d18c1aaff5cd/users/bf364376562742938701ecacca0b39b8/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.173543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb9694-0838-4c46-a0cd-3c98e98694f7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.180137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.181513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.181952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.182589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] PATH_INFO: `/v3/projects/cee8ced67bdf4b5ca8aa12a8940b9bea/users/e5e99d5484594d13b113f27e77b37efd/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.186099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1737469298f94e1b86b7d18c1aaff5cd, user_id=bf364376562742938701ecacca0b39b8, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.189402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.203250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee8ced67bdf4b5ca8aa12a8940b9bea, user_id=e5e99d5484594d13b113f27e77b37efd, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.206743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a32b7-8e57-4a62-879a-b3eb7756144e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.213416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 801/1602] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:41 2022] PUT /identity/v3/projects/1737469298f94e1b86b7d18c1aaff5cd/users/bf364376562742938701ecacca0b39b8/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.214923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.232349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 802/1603] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:41 2022] PUT /identity/v3/projects/cee8ced67bdf4b5ca8aa12a8940b9bea/users/e5e99d5484594d13b113f27e77b37efd/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.237054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53cc2-79ab-42ef-a736-09f451693f48 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.237495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53cc2-79ab-42ef-a736-09f451693f48 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.237816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef53cc2-79ab-42ef-a736-09f451693f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.255464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-cef53cc2-79ab-42ef-a736-09f451693f48 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.257490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cef53cc2-79ab-42ef-a736-09f451693f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.298772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7dd5e-9970-4e89-9707-8b294a38d628 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.324563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 803/1604] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.325916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.326201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.326508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.354063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] MFA Rules not processed for user `8835c23d3ec04b9fa91442d34ce0f84e`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.355455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.373339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.374231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.374549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.374823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] PATH_INFO: `/v3/projects/34e1e9e677d14642bb533001aa8ac301/users/44f56529ce40493f9f6e1da0de9cfca7/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.394214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e1e9e677d14642bb533001aa8ac301, user_id=44f56529ce40493f9f6e1da0de9cfca7, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.397790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e58a5-1d17-49b5-b644-79ac4c75a6bf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.434314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 802/1605] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:41 2022] PUT /identity/v3/projects/34e1e9e677d14642bb533001aa8ac301/users/44f56529ce40493f9f6e1da0de9cfca7/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.435865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113251dd-7b69-4c7a-ab18-80c8c0cbf9a8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.436286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113251dd-7b69-4c7a-ab18-80c8c0cbf9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.436817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113251dd-7b69-4c7a-ab18-80c8c0cbf9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.446957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 804/1606] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.448024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.462536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3792e-d7c3-4dc1-a4a6-733fee0ac249 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.474782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-113251dd-7b69-4c7a-ab18-80c8c0cbf9a8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.476568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-113251dd-7b69-4c7a-ab18-80c8c0cbf9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.478868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.492135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.546807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.547746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.548030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.548302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] PATH_INFO: `/v3/projects/faccee69d2444c61bb9bedff14fe9a25/users/f9383bc17a254fe69d495478dd47230c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.565556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=faccee69d2444c61bb9bedff14fe9a25, user_id=f9383bc17a254fe69d495478dd47230c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.569006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a214e-71da-4cfb-9124-d7ff3878f79d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.574399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 803/1607] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.575951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6862e-096c-4a23-a4b5-d6e98d310a61 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.576341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6862e-096c-4a23-a4b5-d6e98d310a61 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.576752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6862e-096c-4a23-a4b5-d6e98d310a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.588898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 805/1608] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:41 2022] PUT /identity/v3/projects/faccee69d2444c61bb9bedff14fe9a25/users/f9383bc17a254fe69d495478dd47230c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:41.603152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.603465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.603713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.614523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-96d6862e-096c-4a23-a4b5-d6e98d310a61 None None] MFA Rules not processed for user `bf364376562742938701ecacca0b39b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.615966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-96d6862e-096c-4a23-a4b5-d6e98d310a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.622990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.624093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.694933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 806/1609] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.695835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.711448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 804/1610] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.713690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.714132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.714954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.748537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] MFA Rules not processed for user `44f56529ce40493f9f6e1da0de9cfca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.750307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.765660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911d92-0881-43ef-ab13-24ebc8495378 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.769498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.770375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.771107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.771304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.782109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.783690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.795824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 807/1611] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:41.796715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.831683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 805/1612] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:41.833662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:41.844455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.856159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.872236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8a4e7-8a40-4494-b793-bf351388a234 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.880166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578bd49-b8fb-4788-ab26-8443b09eb5a4 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:41.885149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.886104 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.886635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.886755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:41.887429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.887549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.887904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.888181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.911388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:41.912993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135498da-f4dd-4530-898f-ae4ca4a1040e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:41.923483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 806/1613] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:41.924755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1616a-50a8-478f-a179-180be004efaa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:41.925015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1616a-50a8-478f-a179-180be004efaa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:41.925289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1616a-50a8-478f-a179-180be004efaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:41.955559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e9f1616a-50a8-478f-a179-180be004efaa None None] MFA Rules not processed for user `f9383bc17a254fe69d495478dd47230c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:41.958111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9f1616a-50a8-478f-a179-180be004efaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:41.970957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:41.999998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.002209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.015616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 808/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:42.016911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.026946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f4bf2-b9bf-42ab-95f0-1a0e3401aadd service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.031294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.032793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.033167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.033435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.047249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.048976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.062210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 809/1615] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:42.063135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.076405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b8fc5-cef1-461c-8ace-cc16b58f8b12 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.083280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.084424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.085029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.085457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.086469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.089142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.105288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 807/1616] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:42.127314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 810/1617] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:42 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:42.127545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7078a9-ffe3-437c-ad2c-e06345214526 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.129523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7078a9-ffe3-437c-ad2c-e06345214526 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.130245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7078a9-ffe3-437c-ad2c-e06345214526 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.130755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.138639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 808/1618] 10.222.0.49 () {54 vars in 961 bytes} [Thu Jun 30 13:05:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:05:42.146154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.146154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.146762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.210488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:42.215040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:42.219120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca8bf3-5f85-4f7e-9b6a-c55a2fbbb9ab admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.225733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.227779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.228179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.228548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.230129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.233230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df42bc3-df64-4783-93b6-ec1c1c1acad0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.268733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 811/1619] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:42 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:42.269539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174010b7-1cef-429b-be4e-ee887090da26 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.270122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174010b7-1cef-429b-be4e-ee887090da26 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.270590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174010b7-1cef-429b-be4e-ee887090da26 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.273470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 812/1620] 10.222.0.49 () {54 vars in 961 bytes} [Thu Jun 30 13:05:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:05:42.274488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174010b7-1cef-429b-be4e-ee887090da26 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.307089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174010b7-1cef-429b-be4e-ee887090da26 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.315807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.316840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.317312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.317642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.318572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.321359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249f8c3-15bb-46fc-a1cb-8311c8831781 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.340223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 813/1621] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:42.342007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.342413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.342782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.369427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:42.371928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:42.417617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 809/1622] 10.222.0.49 () {58 vars in 1058 bytes} [Thu Jun 30 13:05:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:42.418520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.443792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371959d0-7f26-4f0f-804b-b729f769dd09 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.454534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:42.455688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.466738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.467023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.467301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] PATH_INFO: `/v3/projects/69c8763cf61544d284557a6a54cc5928/users/e1428c50f086415baf3eea9ddd2c6c81/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.517523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c8763cf61544d284557a6a54cc5928, user_id=e1428c50f086415baf3eea9ddd2c6c81, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.524718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.529082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 814/1623] 10.222.0.49 () {58 vars in 1058 bytes} [Thu Jun 30 13:05:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:42.530068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.545792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd6ee3-32fb-4842-938b-0452219ab886 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.557737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.559260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.559977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.560413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.571981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 810/1624] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:42 2022] PUT /identity/v3/projects/69c8763cf61544d284557a6a54cc5928/users/e1428c50f086415baf3eea9ddd2c6c81/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:42.572988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.744908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.748067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33076827-3c64-4d36-897f-76a6f65b28ae admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.748520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.763649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.764428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.764616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.764890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.781210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 815/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:42.783800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:42.792184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:42.879038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce3b2e-f156-4462-8ba5-44f9b1cba032 service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:42.887893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.889813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.916283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 811/1626] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:42.973177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:42.974526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:42.974921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:42.975548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:42.976669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:42.978936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfcb4f2-7110-4a1c-83d1-ef548969ef9c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:42.997329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 816/1627] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:43.001518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:43.020832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08500e-0506-43c0-8484-af6866f603e4 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:43.028612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:43.029821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.030280 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.031872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] PATH_INFO: `/v3/projects/69c8763cf61544d284557a6a54cc5928/users/e1428c50f086415baf3eea9ddd2c6c81/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.036156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c8763cf61544d284557a6a54cc5928, user_id=e1428c50f086415baf3eea9ddd2c6c81, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:43.040596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80daf5e2-3017-4a39-bfc5-bc8bdbe1d046 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:43.056874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 812/1628] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:43 2022] PUT /identity/v3/projects/69c8763cf61544d284557a6a54cc5928/users/e1428c50f086415baf3eea9ddd2c6c81/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:43.068627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.068937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.069421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.145623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] MFA Rules not processed for user `e1428c50f086415baf3eea9ddd2c6c81`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:43.147881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:43.315663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:43.348543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6653f3-48bf-403c-ba7f-cf6a9a4b2e21 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.348927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6653f3-48bf-403c-ba7f-cf6a9a4b2e21 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.349151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6653f3-48bf-403c-ba7f-cf6a9a4b2e21 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.350961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 813/1629] 10.222.0.49 () {54 vars in 912 bytes} [Thu Jun 30 13:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 13:05:43.360214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822a15-08f3-4c64-bdc4-0106236f7158 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.360867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822a15-08f3-4c64-bdc4-0106236f7158 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.360867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822a15-08f3-4c64-bdc4-0106236f7158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.393748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 817/1630] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:43.403519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:43.409115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bf822a15-08f3-4c64-bdc4-0106236f7158 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:43.414875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf822a15-08f3-4c64-bdc4-0106236f7158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:43.531727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 814/1631] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:05:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:05:43.533869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d742-6b1c-41ce-95b0-04d91a01e1c2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.534215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d742-6b1c-41ce-95b0-04d91a01e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.534486 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2d742-6b1c-41ce-95b0-04d91a01e1c2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.537018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 815/1632] 10.222.0.49 () {54 vars in 912 bytes} [Thu Jun 30 13:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:05:43.541866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2d742-6b1c-41ce-95b0-04d91a01e1c2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:43.577330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890d738-f1e1-4097-87db-3beec0b9b51b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:43.587269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:43.588012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.588367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.589700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.612024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:43.615396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de495c9-e86c-4044-9c56-63e7515c8695 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:43.645616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 818/1633] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:43.647144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.647435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.647715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.680113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:43.680824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2d742-6b1c-41ce-95b0-04d91a01e1c2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:43.681531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:43.687130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:43.687814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.688100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.688382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.691857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 819/1634] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:05:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:43.711366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:43.719193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:43.739778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 816/1635] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:43.741193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:43.751305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed97f8-5464-4dd7-ae30-cba8e74bf874 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:43.758085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:43.759598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:43.760130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:43.760737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:43.789243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:43.790792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ea936-29a0-4859-bac6-e2fce853c862 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:43.815443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 820/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:05:44.034349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:44.116426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:44.159756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0966481-d3df-4abe-ae54-c22c5e9f8b9e service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:44.170273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:44.171304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:44.171655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:44.172242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] PATH_INFO: `/v3/users/f9383bc17a254fe69d495478dd47230c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:44.173844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9383bc17a254fe69d495478dd47230c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:44.180378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:44.180378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 30 13:05:44.180378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: Jun 30 13:05:44.182070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:44.355666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9383bc17a254fe69d495478dd47230c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.392389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9383bc17a254fe69d495478dd47230c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.392892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9383bc17a254fe69d495478dd47230c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.443205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9383bc17a254fe69d495478dd47230c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.444201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9383bc17a254fe69d495478dd47230c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.490650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 817/1637] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:05:44 2022] DELETE /identity/v3/users/f9383bc17a254fe69d495478dd47230c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:44.496466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:44.497065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:44.497065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:44.556758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:44.558085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:44.683733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:44.685827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:44.695089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 821/1638] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:05:44.711040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd8519-4232-4088-ad84-abb48093766e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:44.712578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:44.715446 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:44.719478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:44.720550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:44.720837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:44.721150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] PATH_INFO: `/v3/users/bf364376562742938701ecacca0b39b8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:44.725983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf364376562742938701ecacca0b39b8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:44.728149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:44.821578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca0f83-ca20-4ddb-bc13-0bcdaa457199 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:44.827300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:44.828510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:44.828802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:44.829078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:44.842444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf364376562742938701ecacca0b39b8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.843437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:44.846937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2976d-32c5-4658-bb2f-8a58a2727f06 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:44.858304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf364376562742938701ecacca0b39b8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.859061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf364376562742938701ecacca0b39b8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.873225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 822/1639] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:44.880511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf364376562742938701ecacca0b39b8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.881262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf364376562742938701ecacca0b39b8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:44.898393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 818/1640] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:05:44 2022] DELETE /identity/v3/users/bf364376562742938701ecacca0b39b8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:44.900180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:44.900490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:44.900771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:44.927562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:44.929090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:44.939696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:45.033684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 823/1641] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:45.104988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:45.107615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:45.176870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:45.462426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae904-d895-4fda-841e-38fe0c864295 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:45.475713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:45.481084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:45.481802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:45.482108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:45.511008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:45.539435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f119098-8e7a-4209-8283-fe09a8e0e46d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:45.587922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 819/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:45.961205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:45.962748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:46.000858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40613a-aab3-4c41-a7ea-e4fe1cbbf1ba None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.015935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.017138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.017588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.018011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] PATH_INFO: `/v3/projects/faccee69d2444c61bb9bedff14fe9a25` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.021311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faccee69d2444c61bb9bedff14fe9a25)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.024114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: WARNING py.warnings [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 13:05:46.024114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: warnings.warn(msg) Jun 30 13:05:46.024114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: Jun 30 13:05:46.029516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:46.189736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faccee69d2444c61bb9bedff14fe9a25', 'request_id': 'req-a072b57e-a25b-47fc-9afd-16fa42abe608'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:46.212930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faccee69d2444c61bb9bedff14fe9a25', 'request_id': 'req-a072b57e-a25b-47fc-9afd-16fa42abe608'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:46.236578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faccee69d2444c61bb9bedff14fe9a25', 'request_id': 'req-a072b57e-a25b-47fc-9afd-16fa42abe608'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:46.253172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] The token cache is being invalidate because project faccee69d2444c61bb9bedff14fe9a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:05:46.253736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:46.254875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a072b57e-a25b-47fc-9afd-16fa42abe608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:46.258779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 824/1643] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:05:45 2022] DELETE /identity/v3/projects/faccee69d2444c61bb9bedff14fe9a25 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:46.263626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.264194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.264305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.306566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:46.307966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:46.462260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 820/1644] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:46.470149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.470929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.471356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.510277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:46.511765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:46.534711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 825/1645] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:46.536126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:46.553614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afe0ec-eabf-43fd-815d-8f3fcbb98196 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.562871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.563554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.564024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.564297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.565036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:46.603926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.603926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:46.603926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 821/1646] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:46.609382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:46.622673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59b39-ea5d-4e7f-a24d-4487745b52e2 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.629907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.631101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.631426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.631837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.633254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.635823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:46.681888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 826/1647] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:46 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:46.685708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:46.706955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f308b-5a06-478a-9eff-68e0a6b3df81 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.851892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.855477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.855932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.856398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.857414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.862897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:46.899792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 822/1648] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:46 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:46.903771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:46.917179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5619d-082e-4f64-8f2a-2fc29a551f6f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.923564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.924532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.925934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.927180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.928082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.929862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:46.949474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 827/1649] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:46.952511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:46.970357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69a5974-eaa0-4d49-8081-267d1df7ede8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:46.979113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:46.981392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:46.981798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:46.982433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:46.985445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:46.987539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77ef1f-5d31-4b03-8a9c-4aec1d4929ca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.009586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 823/1650] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:47.014368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.039001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324501de-2bcf-447a-bca8-1768cf7f0c0e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:47.047093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:47.048155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.048495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.048842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] PATH_INFO: `/v3/projects/3fabb68db1c34225a34a9fab66811466/users/629006488ba6427992ce6f1be8a88237/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.072528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fabb68db1c34225a34a9fab66811466, user_id=629006488ba6427992ce6f1be8a88237, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:47.076456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.112541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.113173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.113774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.114784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 828/1651] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:47 2022] PUT /identity/v3/projects/3fabb68db1c34225a34a9fab66811466/users/629006488ba6427992ce6f1be8a88237/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:47.151910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] MFA Rules not processed for user `629006488ba6427992ce6f1be8a88237`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:47.153585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:47.188090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:47.317469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 824/1652] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:05:47.332816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.438519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:47.515954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.517426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:47.521465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaaead-f91e-4444-bf4b-c486903226a1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:47.543291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:47.543618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d8d9c-5ce9-4f38-842d-4c8c4a256bfe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:47.544514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.550920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.551279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.552414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:47.554604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.556849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:47.557756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.558465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.558586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] PATH_INFO: `/v3/projects/1737469298f94e1b86b7d18c1aaff5cd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.566612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=1737469298f94e1b86b7d18c1aaff5cd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:47.568924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.613304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 829/1653] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:47 2022] POST /identity/v3/projects => generated 379 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:47.614523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.702562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1737469298f94e1b86b7d18c1aaff5cd', 'request_id': 'req-e363f4d0-c29c-444a-920b-c73aab68c3ed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:47.728855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1737469298f94e1b86b7d18c1aaff5cd', 'request_id': 'req-e363f4d0-c29c-444a-920b-c73aab68c3ed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:47.739615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1737469298f94e1b86b7d18c1aaff5cd', 'request_id': 'req-e363f4d0-c29c-444a-920b-c73aab68c3ed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:47.770176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a5404-eac8-4412-aaf2-8df7d21dd2cc admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:47.779778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] The token cache is being invalidate because project 1737469298f94e1b86b7d18c1aaff5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:05:47.780072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:47.781203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:47.783155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 825/1654] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:05:47 2022] DELETE /identity/v3/projects/1737469298f94e1b86b7d18c1aaff5cd => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:05:47.786879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.791728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:47.898043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:47.898944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.899324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.900916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.900916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:47.903254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.946020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363f4d0-c29c-444a-920b-c73aab68c3ed admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:47.950636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 830/1655] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:47 2022] POST /identity/v3/users => generated 296 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:47.958227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:47.959670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:47.961107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:47.961394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:47.965400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:47.970312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] PATH_INFO: `/v3/users/e1428c50f086415baf3eea9ddd2c6c81` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:47.972319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1428c50f086415baf3eea9ddd2c6c81)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:47.974676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:47.996070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7b2e-9e06-42a8-aa1a-1002fd549545 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:48.010898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:48.011582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:48.011957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:48.012335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:48.013232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:48.015296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:48.047244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 831/1656] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:48.053545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:48.084447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1428c50f086415baf3eea9ddd2c6c81'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:48.088758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:48.102199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1428c50f086415baf3eea9ddd2c6c81'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:48.102876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1428c50f086415baf3eea9ddd2c6c81'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:48.103676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2aca3-7bf2-404c-b17b-8fd876aa17dd admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:48.128114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:48.129296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:48.129714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:48.130158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] PATH_INFO: `/v3/projects/a418f60cfed64db1a101160bc03c7ddd/users/d5e2e4d5388f4daba4e57ffb35b875cc/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:48.131009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1428c50f086415baf3eea9ddd2c6c81'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:48.131957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1428c50f086415baf3eea9ddd2c6c81'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:48.160540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 826/1657] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:05:47 2022] DELETE /identity/v3/users/e1428c50f086415baf3eea9ddd2c6c81 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:48.171102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a418f60cfed64db1a101160bc03c7ddd, user_id=d5e2e4d5388f4daba4e57ffb35b875cc, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:48.173921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:48.210426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 832/1658] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:48 2022] PUT /identity/v3/projects/a418f60cfed64db1a101160bc03c7ddd/users/d5e2e4d5388f4daba4e57ffb35b875cc/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:48.214896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:48.217133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:48.368216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e50ea-a048-4724-88d6-e546a95e6fa9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:48.519780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:48.520929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:48.521221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:48.521622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:48.522408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:48.524605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428601-2f25-4b55-9d29-84689dae3d02 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:48.554848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:48.560496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 827/1659] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:48.575018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c25bf1-6f5d-46ba-8a4d-f2099ad93d43 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:48.580169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:48.581171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:48.581362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:48.581634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] PATH_INFO: `/v3/projects/a418f60cfed64db1a101160bc03c7ddd/users/d5e2e4d5388f4daba4e57ffb35b875cc/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:48.589607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a418f60cfed64db1a101160bc03c7ddd, user_id=d5e2e4d5388f4daba4e57ffb35b875cc, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:48.592428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:48.640778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 833/1660] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:48 2022] PUT /identity/v3/projects/a418f60cfed64db1a101160bc03c7ddd/users/d5e2e4d5388f4daba4e57ffb35b875cc/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:05:48.643001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:48.643216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:48.644087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:48.705353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] MFA Rules not processed for user `d5e2e4d5388f4daba4e57ffb35b875cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:48.706710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:48.894405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:48.933452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 828/1661] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:48.938220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:48.940101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:48.961265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:49.151636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3d22b-e191-4548-ac5a-42eabefc1cf1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:49.167189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:49.168211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:49.169295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:49.169735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:49.192153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:49.196090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:49.229514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 834/1662] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:49.426875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:49.440294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41815e3-22d4-4f95-8deb-338251ac302d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:49.450474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:49.451330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:49.451433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:49.451700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:49.459039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:49.593968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:49.595783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1309c4-06f1-4f3a-b39c-3ffb6ca4f98f service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:49.608976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 829/1663] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:51.184348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:51.186588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:51.315534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf36c42-34b2-4ec8-afc3-e7df0e43f42a service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:51.327215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:51.328254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:51.328531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:51.328801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] PATH_INFO: `/v3/projects/69c8763cf61544d284557a6a54cc5928` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:51.331252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c8763cf61544d284557a6a54cc5928)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:51.333458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:51.446521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c8763cf61544d284557a6a54cc5928', 'request_id': 'req-7855d3ee-e06e-4d24-82eb-77dd5a083330'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:51.470633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c8763cf61544d284557a6a54cc5928', 'request_id': 'req-7855d3ee-e06e-4d24-82eb-77dd5a083330'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:51.480603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c8763cf61544d284557a6a54cc5928', 'request_id': 'req-7855d3ee-e06e-4d24-82eb-77dd5a083330'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:51.500542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] The token cache is being invalidate because project 69c8763cf61544d284557a6a54cc5928 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:05:51.500989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:51.501838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-7855d3ee-e06e-4d24-82eb-77dd5a083330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:51.506961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 835/1664] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:05:51 2022] DELETE /identity/v3/projects/69c8763cf61544d284557a6a54cc5928 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:51.511854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:51.512386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:51.512475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:51.536144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:51.537744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:51.608155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 830/1665] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:51.616136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:51.616582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:51.617063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:51.650976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:51.652426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:51.666121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 836/1666] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:51.670166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:51.693628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bdbbe-f2f9-4e4e-a2a7-b18dc4ce77b6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:51.706950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:51.708785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:51.709846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:51.710190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:51.713107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:51.713983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:51.720294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:51.753170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 831/1667] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:51.757900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:51.771518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8bd4f-0b59-424c-800b-7040b1ebcd04 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:51.779795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:51.781038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:51.781038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:51.781038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:51.781966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:51.784050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:51.850980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 837/1668] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:51 2022] POST /identity/v3/projects => generated 357 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:51.856201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:51.891060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd973df-aed2-46aa-bf3c-9b779c85ef02 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:52.046991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:52.047809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:52.048103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:52.048366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:52.049165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:52.051906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:52.125416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 832/1669] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:51 2022] POST /identity/v3/users => generated 279 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:52.130069 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:52.146787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de74cf3-b5c6-4788-91d1-79fd0c43e631 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:52.154383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:52.155749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:52.156204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:52.156644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:52.157636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:52.159709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:52.171393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:52.187717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 838/1670] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:52.192040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:52.206308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7408c-2e24-4cc4-81e2-5169792288e3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:52.217248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:52.218140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:52.218561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:52.219805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:52.220741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:52.224148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92565a0-cf4d-42ca-bc3e-9b1e63a1842e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:52.246935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:52.257085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 833/1671] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:52.259254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:52.295882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e5911-a80f-4dd0-afcb-6e7b7b0d15ac admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:52.309015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:52.322875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:52.333618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:52.335281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] PATH_INFO: `/v3/projects/0715fbf81eb54d998ef113856fcc317f/users/3cb4bcc4e9c64406a8d7b13151bc05e8/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:52.406008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0715fbf81eb54d998ef113856fcc317f, user_id=3cb4bcc4e9c64406a8d7b13151bc05e8, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:52.412431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:52.444253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 839/1672] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:52 2022] PUT /identity/v3/projects/0715fbf81eb54d998ef113856fcc317f/users/3cb4bcc4e9c64406a8d7b13151bc05e8/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:52.456457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:52.457006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:52.457481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:52.533886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] MFA Rules not processed for user `3cb4bcc4e9c64406a8d7b13151bc05e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:52.536751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:52.771904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 834/1673] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:52 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:52.776401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:52.995032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c2bb6-aac6-4d23-939a-5641cf512fd1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:53.012812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:53.016254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:53.018810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:53.022906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:53.024394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:53.031762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:53.176970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 840/1674] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:52 2022] POST /identity/v3/projects => generated 357 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:53.180741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:53.201080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f472-4a4b-4f9c-a00d-e6f408f57640 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:53.487300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:53.488028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:53.488322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:53.489088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:53.489489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:53.504871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:53.576138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 835/1675] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:53 2022] POST /identity/v3/users => generated 285 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:53.578007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:53.595557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33024f-437c-4d27-8cee-4e20b3a4a377 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:53.618840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:53.620456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:53.620872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:53.621401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:53.622183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:53.623892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:53.679364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 841/1676] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:53.683583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:53.699459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cf116-8df4-4172-9a73-11c80562feaf admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:53.716670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:53.717888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:53.718382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:53.718934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] PATH_INFO: `/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0/users/ea7c6f0ebff946f69438ee280c613ff8/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:53.768780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae8726c4cca403dbbf4f61ec118cdb0, user_id=ea7c6f0ebff946f69438ee280c613ff8, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:53.789373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:53.833628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 836/1677] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:53 2022] PUT /identity/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0/users/ea7c6f0ebff946f69438ee280c613ff8/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:53.840445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:53.870063 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374df82-efe1-4c11-815f-b898552c5a62 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:54.124893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:54.125798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.126085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.126417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.127330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:54.129825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a5244-63b0-4ab0-993f-527bf4b4c4ed admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:54.161534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 842/1678] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:54.165739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:54.177273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224bbf5-e033-4aa2-ad14-947551cfb414 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:54.190887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:54.191977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.192356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.192743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] PATH_INFO: `/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0/users/ea7c6f0ebff946f69438ee280c613ff8/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.194745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.195027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.195297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.204452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae8726c4cca403dbbf4f61ec118cdb0, user_id=ea7c6f0ebff946f69438ee280c613ff8, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:54.213823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35314966-4e9a-43f7-93cd-8994dde8c752 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:54.249340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 837/1679] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:54 2022] PUT /identity/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0/users/ea7c6f0ebff946f69438ee280c613ff8/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:54.253259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] MFA Rules not processed for user `126e2b6fc7114ec3bdc1bb3a99fcf752`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:54.254762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:54.259924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.260389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.260947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.340704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] MFA Rules not processed for user `ea7c6f0ebff946f69438ee280c613ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:54.342725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:54.389220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:54.450293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:54.472146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 843/1680] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:54.493035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:54.494693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:54.556158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 838/1681] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:54 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:54.569960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:54.572076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:54.702953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e0683-5c98-48b0-93b6-d86a8d8cf81e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:54.723280 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:54.724070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.724352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.724665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.765062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:54.766650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:54.779809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86643c85-2b93-4031-98fb-be7969b53d6a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:54.799402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:54.802225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:54.803071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 839/1682] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:54 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:05:54.817196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:54.817982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:54.858942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:54.879482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:54.935798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 844/1683] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:05:55.018602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:55.020497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:55.044145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08631cfe-0adf-40da-af64-4027151e54c4 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:55.056695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:55.057581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:55.058012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:55.058476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:55.216973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:55.218700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658a95c-3eb5-48b1-9495-c9d655a41ba9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:55.231682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 840/1684] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:56.327916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:56.329405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:56.426450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a424382-ec3c-4ef4-a606-9b74b4988803 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:56.431609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a424382-ec3c-4ef4-a606-9b74b4988803 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:56.432191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a424382-ec3c-4ef4-a606-9b74b4988803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:56.486569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3a424382-ec3c-4ef4-a606-9b74b4988803 None None] MFA Rules not processed for user `2e84715279114df09ecfe4335c926066`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:56.488640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a424382-ec3c-4ef4-a606-9b74b4988803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:56.500651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ca3ad-1a2a-4216-aa77-5834c10396b4 service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:56.510895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:56.513311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:56.514638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:56.515359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] PATH_INFO: `/v3/users/44f56529ce40493f9f6e1da0de9cfca7` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:56.518130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f56529ce40493f9f6e1da0de9cfca7)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:56.521431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:56.628262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f56529ce40493f9f6e1da0de9cfca7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:56.641568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f56529ce40493f9f6e1da0de9cfca7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:56.643511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f56529ce40493f9f6e1da0de9cfca7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:56.653636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:56.663527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:56.673614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f56529ce40493f9f6e1da0de9cfca7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:56.673614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c4ac73dd-384c-4038-80f0-50cc297f337d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f56529ce40493f9f6e1da0de9cfca7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:56.674394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 841/1685] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:56.676099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:56.676576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:56.677628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:56.707093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 845/1686] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:05:56 2022] DELETE /identity/v3/users/44f56529ce40493f9f6e1da0de9cfca7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:56.712015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:56.712615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:56.713083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:56.716294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] MFA Rules not processed for user `2e84715279114df09ecfe4335c926066`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:56.720126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:56.743436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:56.745050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:56.837341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 846/1687] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:05:56.850169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:56.852611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:56.896357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 842/1688] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:56.909197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:56.911505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:57.090304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42185943-02d1-48ea-ba4b-324d55643afa None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:57.114008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:57.116638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:57.116984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:57.117181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:57.142169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:57.143697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608b27c-9e27-436f-b3db-47e845bce57f service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:57.179458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88dbe9-e6a1-41d5-934e-772a45e4155a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:57.182131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 847/1689] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:57.185648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90c4a8-8946-4207-9c07-901436c8f21f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:57.186249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90c4a8-8946-4207-9c07-901436c8f21f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:57.186561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90c4a8-8946-4207-9c07-901436c8f21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:57.193692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:57.194381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:57.194670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:57.194948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:57.226934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:57.230183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:57.248311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 843/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:57.283224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3c90c4a8-8946-4207-9c07-901436c8f21f None None] MFA Rules not processed for user `e5e99d5484594d13b113f27e77b37efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:57.288940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3c90c4a8-8946-4207-9c07-901436c8f21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:57.517568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 848/1691] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:57.651868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:57.654131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:57.680664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa2d85-b644-4845-a1b8-70ead8e09ee5 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:57.691243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:57.693048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:57.693574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:57.694060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:57.706347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:57.718192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171e903-e643-4909-b0c9-e1d70f324ead service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:57.733983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 844/1692] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:58.520133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79f541-c07e-47ef-b70b-ad39ad5c87b1 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:58.520736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79f541-c07e-47ef-b70b-ad39ad5c87b1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:58.520827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79f541-c07e-47ef-b70b-ad39ad5c87b1 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:58.524565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 849/1693] 10.222.0.49 () {54 vars in 912 bytes} [Thu Jun 30 13:05:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 13:05:58.532317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:58.532849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:58.534467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:58.588883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:58.589681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:58.590678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:58.693952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:58.697928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:58.703081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:58.711423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] MFA Rules not processed for user `e0df4b789a804a00a267bcc5cf778324`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:58.717293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:58.728817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 845/1694] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:05:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:58.730003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:58.734905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:58.768348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e2d4-3042-4f6e-8c58-5af29b632001 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:58.783909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:58.787083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:58.788370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:58.789284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] PATH_INFO: `/v3/projects/34e1e9e677d14642bb533001aa8ac301` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:58.799555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e1e9e677d14642bb533001aa8ac301)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:58.805549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:58.945120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e1e9e677d14642bb533001aa8ac301', 'request_id': 'req-cc938d36-72b8-49be-a508-0c0189242f86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:58.947597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:58.962513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e1e9e677d14642bb533001aa8ac301', 'request_id': 'req-cc938d36-72b8-49be-a508-0c0189242f86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:58.975314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e1e9e677d14642bb533001aa8ac301', 'request_id': 'req-cc938d36-72b8-49be-a508-0c0189242f86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:58.999742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] The token cache is being invalidate because project 34e1e9e677d14642bb533001aa8ac301 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:05:59.000043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:59.000731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc938d36-72b8-49be-a508-0c0189242f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:05:59.002998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 850/1695] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:59.003992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.014653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 846/1696] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:05:58 2022] DELETE /identity/v3/projects/34e1e9e677d14642bb533001aa8ac301 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:05:59.016191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.016538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.016824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.047509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bec7c2-d96c-457f-9757-2e589de78a05 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.069617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:59.070908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:59.208082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.210751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.211222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.211642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.224638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:59.279947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.281653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c586-d057-4c65-85c7-268eeff90667 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.302621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 847/1697] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:59.302621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.302621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:59.314055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 851/1698] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:59.315794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.315938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.317079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.393934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:59.398286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:59.421202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 852/1699] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:05:59.428156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.456746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8205853-dc55-4498-bfdf-e54936709ae9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.469668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.470609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.471047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.471456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.472349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:05:59.472834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.474956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.493643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 853/1700] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:05:59.494911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.506546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90120-426d-429a-a14f-aa8913f63563 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.512785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.514690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.515159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.515623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.516474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.518699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.547628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad19ad-e21c-44b6-abb4-fef2eccbabde None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.557950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.558852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.559427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.560036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.564081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:05:59.577163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 854/1701] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:05:59 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:59.578162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.689116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d244f-049d-4168-a487-65948e78ac74 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.695803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.696668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.697193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.697612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.699011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.701435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.712359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:59.717797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.719726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.739803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 855/1702] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:05:59 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:05:59.744225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.747599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 848/1703] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:05:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:05:59.757867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49a140-4b7d-4ccb-af52-db250d3027bb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.766695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.767333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.767695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.768040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.768866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.770811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.794233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 856/1704] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:05:59.798632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.816328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c149a11-15ae-49f5-ac2e-dae293f95c4b service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.825808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.826612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.827396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.827797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.828723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.831091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd005d-059b-40f2-856f-71b4b0449de6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.865830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:05:59.866035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 849/1705] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:05:59.887489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39b5bd-8fb0-482d-854c-1f1d66a1cd16 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:05:59.912047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:05:59.913542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.914062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.914571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] PATH_INFO: `/v3/projects/d839e9f8f73f4f0283f59c1fe1fb931b/users/ea28f54fe3c040de8e8b5cd823ae6c14/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.932199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0c2b-3c7a-4231-9902-95b1f6b8cba3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:05:59.932496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0c2b-3c7a-4231-9902-95b1f6b8cba3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:05:59.933094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b0c2b-3c7a-4231-9902-95b1f6b8cba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:05:59.953737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:05:59.967855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d839e9f8f73f4f0283f59c1fe1fb931b, user_id=ea28f54fe3c040de8e8b5cd823ae6c14, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:05:59.970755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49e71b-eaa5-4c94-a894-6fee2c7c9a5b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:05:59.991323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7e4b0c2b-3c7a-4231-9902-95b1f6b8cba3 None None] MFA Rules not processed for user `126e2b6fc7114ec3bdc1bb3a99fcf752`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:05:59.995923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7e4b0c2b-3c7a-4231-9902-95b1f6b8cba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:00.011888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 857/1706] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:05:59 2022] PUT /identity/v3/projects/d839e9f8f73f4f0283f59c1fe1fb931b/users/ea28f54fe3c040de8e8b5cd823ae6c14/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:00.025061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fcacb-de6d-4291-a178-cd8455e167df None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:00.025390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fcacb-de6d-4291-a178-cd8455e167df None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:00.025639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3fcacb-de6d-4291-a178-cd8455e167df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:00.111420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8f3fcacb-de6d-4291-a178-cd8455e167df None None] MFA Rules not processed for user `ea28f54fe3c040de8e8b5cd823ae6c14`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:00.112687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8f3fcacb-de6d-4291-a178-cd8455e167df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:00.180392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 850/1707] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:05:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:00.426811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 858/1708] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:00.600615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd2-af0f-4718-bb53-4f6610d951df None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:00.603245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd2-af0f-4718-bb53-4f6610d951df None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:00.603772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b1bd2-af0f-4718-bb53-4f6610d951df None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:00.606347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 851/1709] 10.222.0.49 () {54 vars in 960 bytes} [Thu Jun 30 13:06:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 13:06:00.614736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:00.615301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:00.615301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:00.655410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:00.656856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:00.832705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 859/1710] 10.222.0.49 () {58 vars in 1057 bytes} [Thu Jun 30 13:06:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:00.843053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b1bd2-af0f-4718-bb53-4f6610d951df None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:00.872997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b1bd2-af0f-4718-bb53-4f6610d951df None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:00.893679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:00.894975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:00.895437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:00.895883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:00.899044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:00.932618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:00.938691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05b740-4413-46f1-add4-dc652ec2cdeb service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:01.006705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 852/1711] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:01.033846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:01.035612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:01.218101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:01.332374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9876a5-14f9-42de-a3ae-90a9347aa7ac None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:01.374189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:01.374919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:01.377415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:01.377711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:01.415593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:01.417569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4465502-e683-4a81-91dc-bb20c5020375 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:01.459459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 860/1712] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:01.750263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871568-2725-48d8-b562-19307be9d730 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:01.750263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871568-2725-48d8-b562-19307be9d730 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:01.750549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33871568-2725-48d8-b562-19307be9d730 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:01.752287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 853/1713] 10.222.0.49 () {54 vars in 912 bytes} [Thu Jun 30 13:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:06:01.761540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0852d-92fa-445f-be51-d1bde4334c15 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:01.762045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0852d-92fa-445f-be51-d1bde4334c15 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:01.762198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d0852d-92fa-445f-be51-d1bde4334c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:01.824354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-62d0852d-92fa-445f-be51-d1bde4334c15 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:01.824911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-62d0852d-92fa-445f-be51-d1bde4334c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:01.974770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 861/1714] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:06:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:01.981258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33871568-2725-48d8-b562-19307be9d730 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:01.983429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-33871568-2725-48d8-b562-19307be9d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:02.008815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33871568-2725-48d8-b562-19307be9d730 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:02.019910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:02.020071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:02.020337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:02.020610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:02.035356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:02.041596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:02.060718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 854/1715] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:03.679850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:03.680437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:03.680437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:03.753341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:03.755603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:04.055066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200abeb-412f-4304-8380-1fc96a231f1a service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:04.082803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:04.084189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:04.094912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:04.095458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] PATH_INFO: `/v3/users/8835c23d3ec04b9fa91442d34ce0f84e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:04.100043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8835c23d3ec04b9fa91442d34ce0f84e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:04.102115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:04.308728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] MFA Rules not processed for user `e5e99d5484594d13b113f27e77b37efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:04.309991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:04.312846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8835c23d3ec04b9fa91442d34ce0f84e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:04.342133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8835c23d3ec04b9fa91442d34ce0f84e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:04.342894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8835c23d3ec04b9fa91442d34ce0f84e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:04.388619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8835c23d3ec04b9fa91442d34ce0f84e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:04.392239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c1f4eda9-7db6-4be5-bb44-1931e55c4d0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8835c23d3ec04b9fa91442d34ce0f84e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:04.455952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 855/1716] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:03 2022] DELETE /identity/v3/users/8835c23d3ec04b9fa91442d34ce0f84e => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:04.457958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:04.458246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:04.458547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:04.488258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:04.541614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:04.543847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:04.651440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 862/1717] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 973 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:04.831338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 856/1718] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:04.922484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:04.924172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:04.949734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35981fc6-7fb7-4e5d-8cc4-bb1bf0bfc07c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:05.139597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:05.140927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.141390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.141801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.158829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:05.160743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:05.161878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:05.163430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96672ff-18b2-44b4-a658-fe5edbaa1bc0 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:05.176766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 863/1719] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:05.187989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834a96a-38b6-4f8c-b950-6f4c751be07c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:05.194530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:05.195768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.196219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.196642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] PATH_INFO: `/v3/users/629006488ba6427992ce6f1be8a88237` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.199153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=629006488ba6427992ce6f1be8a88237)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:05.201948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:05.315298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629006488ba6427992ce6f1be8a88237'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:05.332127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629006488ba6427992ce6f1be8a88237'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:05.333229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629006488ba6427992ce6f1be8a88237'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:05.352680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.353028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.353310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.373822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629006488ba6427992ce6f1be8a88237'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:05.374771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c34cebc-b822-4411-93e2-13d7705285ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629006488ba6427992ce6f1be8a88237'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:05.391173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:05.393699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:05.409183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 857/1720] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:05 2022] DELETE /identity/v3/users/629006488ba6427992ce6f1be8a88237 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:05.432159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.433294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.433582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.467406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:05.490869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:05.493009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:05.536287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 864/1721] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:06:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:05.659551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 858/1722] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:05.681020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:05.682527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:05.728645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:05.730359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:05.740862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e678c-7ed8-42ea-af9c-3ed2222afeec None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:05.770987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:05.912861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257b57a-6d8f-49ed-9c87-213ac54ff0a0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:05.927719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:05.928261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:05.928529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.928816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.929312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.930179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:05.930508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:05.932467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:05.978487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:05.986850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:06.024100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 865/1723] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:06.074711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:06.076625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12412b0-c7a8-4c39-935d-bc8583cac984 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:06.089215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 859/1724] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:06:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:06:06.326285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:06.327726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:06.363784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534e068-0bed-4253-98a6-7c7e0f75651f service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:06.373562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:06.374771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:06.375215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:06.375642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] PATH_INFO: `/v3/users/ea28f54fe3c040de8e8b5cd823ae6c14` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:06.377850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea28f54fe3c040de8e8b5cd823ae6c14)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:06.381874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:06.475229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:06.477086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:06.477534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:06.505676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea28f54fe3c040de8e8b5cd823ae6c14'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:06.519214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea28f54fe3c040de8e8b5cd823ae6c14'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:06.520972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea28f54fe3c040de8e8b5cd823ae6c14'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:06.524024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] MFA Rules not processed for user `e0df4b789a804a00a267bcc5cf778324`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:06.525571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:06.541909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea28f54fe3c040de8e8b5cd823ae6c14'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:06.543025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f29b719d-87e2-44ac-a208-87f50a5929ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea28f54fe3c040de8e8b5cd823ae6c14'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:06.566837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 866/1725] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:06 2022] DELETE /identity/v3/users/ea28f54fe3c040de8e8b5cd823ae6c14 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:06.568876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428009-5ab1-4784-b07b-34852e364707 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:06.569166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428009-5ab1-4784-b07b-34852e364707 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:06.569937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19428009-5ab1-4784-b07b-34852e364707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:06.601954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-19428009-5ab1-4784-b07b-34852e364707 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:06.604054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-19428009-5ab1-4784-b07b-34852e364707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:06.744341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:06.761212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 860/1726] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:06.768667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:06.773760 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:06.791513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 867/1727] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:06.802708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428009-5ab1-4784-b07b-34852e364707 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:06.804544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-19428009-5ab1-4784-b07b-34852e364707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:06.830830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c66fc5-0916-48cb-8a4e-606a10fb1f46 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:06.885426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:06.886837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:06.887329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:06.887688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] PATH_INFO: `/v3/projects/22ec4cf465ec4c01be414c7f55351b20` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:06.898907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ec4cf465ec4c01be414c7f55351b20)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:06.900871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:06.927331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19428009-5ab1-4784-b07b-34852e364707 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:06.935055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:06.936704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:06.937552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:06.938103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:06.952826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:06.955233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2480a-1b51-4b16-8da2-9315b6e0fd7e service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:06.972180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 868/1728] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:07.149718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ec4cf465ec4c01be414c7f55351b20', 'request_id': 'req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:07.170953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ec4cf465ec4c01be414c7f55351b20', 'request_id': 'req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:07.173031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:07.185221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ec4cf465ec4c01be414c7f55351b20', 'request_id': 'req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:07.211296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] The token cache is being invalidate because project 22ec4cf465ec4c01be414c7f55351b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:07.211852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:07.214153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb75b6ef-2ac0-48fb-b9a1-a94a04b8175f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:07.219161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 861/1729] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:06 2022] DELETE /identity/v3/projects/22ec4cf465ec4c01be414c7f55351b20 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:07.225207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:07.226164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:07.227215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:07.315480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:07.320222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:07.550614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 869/1730] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:07.554288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:07.554620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:07.561942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:07.624213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:07.625587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:07.670479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:07.670712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 862/1731] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:07.694545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc4891-312a-46e9-aef4-7debe8ae3e0f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:07.703275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:07.704142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:07.704666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:07.705082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:07.706035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:07.706507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:07.711750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:07.731750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 870/1732] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:07.736408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:07.748626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d3cac-5311-48a8-acd3-1aa23dc0db0e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:07.756371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:07.757283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:07.757771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:07.758240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:07.759339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:07.761971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:07.839923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 863/1733] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:07 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:07.845458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:07.883502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdfd99-5237-4064-bb5a-dc0a9091588f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:08.066754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:08.069291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:08.101754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:08.103238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.103594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.103937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.104731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:08.106849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:08.207856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 871/1734] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:07 2022] POST /identity/v3/users => generated 282 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:08.209030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:08.239071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289ccd8-de93-4534-8656-ef2ce768bb24 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:08.244296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf0188-db01-4045-9cb9-b12d99ffa409 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:08.263283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:08.266806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.269553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.271999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] PATH_INFO: `/v3/projects/d839e9f8f73f4f0283f59c1fe1fb931b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.273405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:08.274192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.274533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.274809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.275791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:08.277884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d839e9f8f73f4f0283f59c1fe1fb931b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:08.280624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:08.283394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:08.285013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:08.309887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 872/1735] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:08.310958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:08.323774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1d5a1-a161-45af-b862-4bd7e7977071 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:08.333279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:08.334001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.334262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.334567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.335618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:08.338139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:08.367590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 873/1736] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:08.374514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:08.417558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03475d-4eca-4b5b-b498-3ac972aae2eb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:08.461063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd839e9f8f73f4f0283f59c1fe1fb931b', 'request_id': 'req-fccded7f-a5bf-43d8-a8b4-82799e2356d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:08.494131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd839e9f8f73f4f0283f59c1fe1fb931b', 'request_id': 'req-fccded7f-a5bf-43d8-a8b4-82799e2356d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:08.510681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd839e9f8f73f4f0283f59c1fe1fb931b', 'request_id': 'req-fccded7f-a5bf-43d8-a8b4-82799e2356d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:08.547521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] The token cache is being invalidate because project d839e9f8f73f4f0283f59c1fe1fb931b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:08.548245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:08.549919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fccded7f-a5bf-43d8-a8b4-82799e2356d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:08.553036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 864/1737] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:08 2022] DELETE /identity/v3/projects/d839e9f8f73f4f0283f59c1fe1fb931b => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:08.558654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501ebc-c27b-42b5-8010-b3bacff90d7a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.559215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501ebc-c27b-42b5-8010-b3bacff90d7a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.560425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4501ebc-c27b-42b5-8010-b3bacff90d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.623244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b4501ebc-c27b-42b5-8010-b3bacff90d7a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:08.627418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b4501ebc-c27b-42b5-8010-b3bacff90d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:08.686113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:08.687406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.687843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.688450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] PATH_INFO: `/v3/projects/ac3bd7ca36cd417ab6e235555d9a85f9/users/3f4a86becf594f00a2bbdcf3a23056b6/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.774831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:08.784354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3bd7ca36cd417ab6e235555d9a85f9, user_id=3f4a86becf594f00a2bbdcf3a23056b6, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:08.788027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b3fcd-b568-41fd-a70b-0d6244a40a2e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:08.816247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 874/1738] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:08 2022] PUT /identity/v3/projects/ac3bd7ca36cd417ab6e235555d9a85f9/users/3f4a86becf594f00a2bbdcf3a23056b6/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:08.830466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.830863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.831068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.851853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 865/1739] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:08.868331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:08.874732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:08.875338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:08.912386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] MFA Rules not processed for user `3f4a86becf594f00a2bbdcf3a23056b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:08.913674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:08.972153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:08.973480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:09.152899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 875/1740] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:09.160075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.161833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:09.208340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 866/1741] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:09.213517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.241210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2109ed-b766-4e31-ad09-e856589a9707 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.253083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.253876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.254177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.271053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.272033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:09.272315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.274091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.302195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 867/1742] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:06:09.308312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.327378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187349e4-bc27-4c7a-8fae-b9fb05919da8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.334394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.335570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.335921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.336260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.337073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.340259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.345753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:09.351119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6d6ae-b6b6-4256-a519-ce7ab0982149 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.365939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.367171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.367552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.368036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.368923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.381034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.442810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 876/1743] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:09 2022] POST /identity/v3/projects => generated 365 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:09.446550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.452094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:09.478139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 868/1744] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:09 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:09.494141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.713140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fdf5af-98cb-4c83-af42-f6b83281ba9d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.723728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.724524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.726979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.726979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.727948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.733391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.796555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 869/1745] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:09 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:09.812816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.812931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194722ba-6a12-45c0-a6b7-f191cd071744 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.829730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.831733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.832084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.832368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] PATH_INFO: `/v3/projects/3fabb68db1c34225a34a9fab66811466` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.836214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fabb68db1c34225a34a9fab66811466)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.850394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.860064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8261277-d44b-40cc-973e-8fb676f792e7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:09.890567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:09.891815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:09.892440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:09.892952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:09.893848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:09.896903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:09.976826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fabb68db1c34225a34a9fab66811466', 'request_id': 'req-e7510071-e329-4d70-83bc-34e50724e0a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:09.990958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 870/1746] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:09 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:09.992968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:09.999451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fabb68db1c34225a34a9fab66811466', 'request_id': 'req-e7510071-e329-4d70-83bc-34e50724e0a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.013316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fabb68db1c34225a34a9fab66811466', 'request_id': 'req-e7510071-e329-4d70-83bc-34e50724e0a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.023402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3f9a0-f654-4c79-acc5-b1b007690f81 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.044795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] The token cache is being invalidate because project 3fabb68db1c34225a34a9fab66811466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:10.050916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.052651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.056365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 877/1747] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:09 2022] DELETE /identity/v3/projects/3fabb68db1c34225a34a9fab66811466 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:10.057447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.069534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:10.113635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:10.280779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.281692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.282212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.283854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.286007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.289475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.323421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7510071-e329-4d70-83bc-34e50724e0a3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.324198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 871/1748] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:10.325511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.327396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:10.356358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.357605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.358103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.358205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46832e56-84f2-4ac5-b5da-828fa62f2ad1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.358878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.360618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.363306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.366531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.367370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.367650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.367928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] PATH_INFO: `/v3/users/d5e2e4d5388f4daba4e57ffb35b875cc` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.369858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e2e4d5388f4daba4e57ffb35b875cc)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.372031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.381786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 878/1749] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:10.383098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.387989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:10.409451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037d91-7316-4bc5-9ee7-4a5e4d52ac77 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.421885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.423030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.423322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.423601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] PATH_INFO: `/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32/users/c8b2a7c4e6f44f9b998dd1c800de2871/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.433179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:10.498360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e2e4d5388f4daba4e57ffb35b875cc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.505171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee53f071ca148eda21ac90aaf3d7f32, user_id=c8b2a7c4e6f44f9b998dd1c800de2871, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.509045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.514690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e2e4d5388f4daba4e57ffb35b875cc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.515802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e2e4d5388f4daba4e57ffb35b875cc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.538532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e2e4d5388f4daba4e57ffb35b875cc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.539690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e2e4d5388f4daba4e57ffb35b875cc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:10.546854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 879/1750] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:10 2022] PUT /identity/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32/users/c8b2a7c4e6f44f9b998dd1c800de2871/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:10.548513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.561341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:10.566354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 872/1751] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:10 2022] DELETE /identity/v3/users/d5e2e4d5388f4daba4e57ffb35b875cc => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:10.566354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.566995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:10.676451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca688261-2fab-4e70-a4c0-4cee3d9c5a23 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.684408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.686302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.686886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.687512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.688869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.691692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.728599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 873/1752] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:10.729667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.750473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d779502a-f1b1-4c0e-9633-254ee2f2ce7c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.761312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4f01c-a2ac-47b3-bae2-003195f91263 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.761733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.762464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.762937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.763045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.763815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.765688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.783354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:10.784085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.784368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.784653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] PATH_INFO: `/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32/users/c8b2a7c4e6f44f9b998dd1c800de2871/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:10.789083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee53f071ca148eda21ac90aaf3d7f32, user_id=c8b2a7c4e6f44f9b998dd1c800de2871, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:10.791874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0afcf-cf9c-4806-addc-25284ca1956f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:10.808666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 880/1753] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:10.812689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:10.842998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 874/1754] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:10 2022] PUT /identity/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32/users/c8b2a7c4e6f44f9b998dd1c800de2871/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:10.848144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0116eb-08ba-4a8e-8fc2-2f493a92856b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:10.855082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0a49-a572-45e2-bf6a-a5154c7dcc17 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:10.856780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0a49-a572-45e2-bf6a-a5154c7dcc17 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:10.857150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe0a49-a572-45e2-bf6a-a5154c7dcc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:11.011809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-adbe0a49-a572-45e2-bf6a-a5154c7dcc17 None None] MFA Rules not processed for user `c8b2a7c4e6f44f9b998dd1c800de2871`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:11.014258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-adbe0a49-a572-45e2-bf6a-a5154c7dcc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:11.039584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:11.163186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:11.163186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:11.163186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:11.163186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] PATH_INFO: `/v3/projects/c6acab5e8aaf4c4e9a9d5a3971256222/users/dea452c43c274562a23b6621870d8631/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:11.224810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6acab5e8aaf4c4e9a9d5a3971256222, user_id=dea452c43c274562a23b6621870d8631, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:11.240253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d3a63-a749-490a-af73-413d24a72f97 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:11.340476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 881/1755] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:10 2022] PUT /identity/v3/projects/c6acab5e8aaf4c4e9a9d5a3971256222/users/dea452c43c274562a23b6621870d8631/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:11.347279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:11.347621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:11.347918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:11.400671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:11.402493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:11.415639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 875/1756] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:11.416209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:11.416209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:11.416739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:11.544283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] MFA Rules not processed for user `dea452c43c274562a23b6621870d8631`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:11.549587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:11.566425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 882/1757] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 30 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:11.567977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:11.569700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:11.591238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:11.652114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7118ce-7560-41f9-8ba8-5deb68516fb3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:11.656584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:11.657304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:11.657636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:11.658092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:11.756216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:11.757901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:11.773352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 883/1758] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:11.802092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:11.802243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 876/1759] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:11.824360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:11.838999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:11.918877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb785be-cb0a-4ae6-9952-5ebc4394227b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:11.956041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:11.959319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:11.959319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:11.959319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:12.029633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:12.038730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:12.091356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 877/1760] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:12.096639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:12.141967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40bd9a-9246-49c5-8614-d4d05656d9a9 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:12.146262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e11440-7220-46a4-a170-215817177f1a service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:12.152327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:12.153845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:12.154911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:12.155647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:12.157171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:12.159563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:12.159708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:12.166311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:12.168060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:12.168548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:12.193302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:12.217610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 884/1761] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:11 2022] POST /identity/v3/projects => generated 383 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:12.222018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:12.566883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:12.579838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:12.591532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:12.650683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 878/1762] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:12.664540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323cf57-f9c5-491a-a2f1-aa266705c4d0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:12.697388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:12.698117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:12.698440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:12.698925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:12.705145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:12.716017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:12.779712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 885/1763] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:12 2022] POST /identity/v3/users => generated 298 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:12.786053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:12.808012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091361c0-5115-437d-a9c7-227505d6f711 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:12.830975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:12.833214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:12.833552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:12.833835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:12.834724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:12.858349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:12.927300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 879/1764] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:12.928732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:12.952070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebd87-daad-4f6a-b576-d3278da5bd8c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:12.982668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:12.989862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:12.991083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:12.994707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] PATH_INFO: `/v3/projects/2db2618a83884e7bbe98af42294212fe/users/c2fdb9e88ecb401d989c692521fd8e05/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:13.081884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db2618a83884e7bbe98af42294212fe, user_id=c2fdb9e88ecb401d989c692521fd8e05, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:13.118441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:13.162839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 886/1765] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:12 2022] PUT /identity/v3/projects/2db2618a83884e7bbe98af42294212fe/users/c2fdb9e88ecb401d989c692521fd8e05/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:13.167351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:13.175460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:13.181758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4215f4-6093-4949-b19a-84a980ea0823 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:13.400723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:13.404233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:13.404704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:13.405633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:13.409097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:13.421882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c709097-50b9-41be-be02-f359ae293739 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:13.445888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 880/1766] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:13.449625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:13.462111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dce69-9402-46d4-906b-dd69843e1ef2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:13.486247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:13.487781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:13.488348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:13.488786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] PATH_INFO: `/v3/projects/2db2618a83884e7bbe98af42294212fe/users/c2fdb9e88ecb401d989c692521fd8e05/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:13.495867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db2618a83884e7bbe98af42294212fe, user_id=c2fdb9e88ecb401d989c692521fd8e05, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:13.514691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:13.557825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 887/1767] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:13 2022] PUT /identity/v3/projects/2db2618a83884e7bbe98af42294212fe/users/c2fdb9e88ecb401d989c692521fd8e05/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:13.571086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:13.571086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:13.571415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:13.636092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] MFA Rules not processed for user `c2fdb9e88ecb401d989c692521fd8e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:13.637365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:13.865138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 881/1768] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:13.878803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:13.890867 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:14.020290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97494-cb6c-4bee-ae3f-f6681da0254a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:14.058481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:14.063548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:14.064504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:14.068683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:14.083299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:14.085343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:14.099751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 888/1769] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:14.424463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:14.445108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8078500-8fac-4181-9fdc-2e169e8c7ae3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:14.455725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:14.458777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:14.459290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:14.459764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:14.671336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:14.689812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f23e1c-deca-4bbc-82d9-e07029d81bf2 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:14.741814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 882/1770] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:15.409624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:15.412200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:15.715292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cfad29-793b-4f15-8581-88e5373b9264 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:15.747787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:15.752168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:15.752488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:15.752774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] PATH_INFO: `/v3/projects/a418f60cfed64db1a101160bc03c7ddd` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:15.757998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a418f60cfed64db1a101160bc03c7ddd)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:15.766815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:15.924769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:15.935089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a418f60cfed64db1a101160bc03c7ddd', 'request_id': 'req-159a2ecc-0be3-436b-b201-a5312c43e8d2'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:15.955757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a418f60cfed64db1a101160bc03c7ddd', 'request_id': 'req-159a2ecc-0be3-436b-b201-a5312c43e8d2'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:15.974057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a418f60cfed64db1a101160bc03c7ddd', 'request_id': 'req-159a2ecc-0be3-436b-b201-a5312c43e8d2'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:16.004127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] The token cache is being invalidate because project a418f60cfed64db1a101160bc03c7ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:16.006243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:16.009403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-159a2ecc-0be3-436b-b201-a5312c43e8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:16.018030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 889/1771] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:15 2022] DELETE /identity/v3/projects/a418f60cfed64db1a101160bc03c7ddd => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:16.019917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:16.020517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:16.020517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:16.077243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:16.079162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:16.125542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:16.234391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 883/1772] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:16.236674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:16.237180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:16.237707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:16.303750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:16.317997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:16.359898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:16.367545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 890/1773] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:16.430881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac48003-6929-4ace-a418-2d49e61969b6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:16.443689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:16.444855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:16.445769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:16.447481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:16.448662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:16.451609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:16.455793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:16.473743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 884/1774] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:06:16.480097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:16.502851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8cc97-5ec7-47d4-8bbd-847090cb6811 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:16.515577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:16.516720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:16.518374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:16.518904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:16.522077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:16.525941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:16.611166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:16.626017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 891/1775] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:16 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:16.639551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e38cdd-c816-4c30-8e1c-d979b03917cf admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:16.840425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:16.842686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:16.843428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:16.847247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:16.849042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:16.851096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:16.951776 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:16.955462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 885/1776] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:16 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:16.988789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed6181-f4fb-40a5-af48-27e52af5dc05 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:17.002941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:17.004425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:17.004938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:17.006919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:17.008411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:17.010584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:17.029785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 892/1777] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:17.035669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:17.052632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901dc34-ee94-423f-865e-b88ea226978c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:17.063588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:17.065728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:17.066838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:17.067236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:17.068245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:17.071139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f93507-9613-448c-9367-ca52995351ea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:17.115231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 886/1778] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:17.115402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:17.128898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad40772-d5d6-4a85-98bb-c648235d8dbe admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:17.140400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:17.141330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:17.141497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:17.141828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] PATH_INFO: `/v3/projects/383fccfcf18341cbbaea31a5154cdfb1/users/65c27fed42a046b1a47e4dd6f8007fe1/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:17.178351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383fccfcf18341cbbaea31a5154cdfb1, user_id=65c27fed42a046b1a47e4dd6f8007fe1, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:17.183432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:17.239441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 893/1779] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:17 2022] PUT /identity/v3/projects/383fccfcf18341cbbaea31a5154cdfb1/users/65c27fed42a046b1a47e4dd6f8007fe1/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:17.253046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1d25b-414d-4d61-b242-68ae43ba1afa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:17.253268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1d25b-414d-4d61-b242-68ae43ba1afa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:17.253566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1d25b-414d-4d61-b242-68ae43ba1afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:17.387099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5cd1d25b-414d-4d61-b242-68ae43ba1afa None None] MFA Rules not processed for user `65c27fed42a046b1a47e4dd6f8007fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:17.394030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5cd1d25b-414d-4d61-b242-68ae43ba1afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:17.765455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 887/1780] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 514 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:17.770437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:17.842942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e04ab7-cb42-4c68-ba1c-65543bd67985 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:17.843217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e04ab7-cb42-4c68-ba1c-65543bd67985 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:17.843491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e04ab7-cb42-4c68-ba1c-65543bd67985 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:17.849199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 888/1781] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:06:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:06:18.002332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea67dd-a3c8-4776-9145-e625fd1fdb12 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.020135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.021187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.021356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.021618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.022356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.024301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ad9c8-0670-4e56-bd2c-21b7e17c8afa admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.102377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e04ab7-cb42-4c68-ba1c-65543bd67985 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:18.103366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 894/1782] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:17 2022] POST /identity/v3/projects => generated 367 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:18.198578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:18.198804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881100-6c54-404e-a855-94b9353460cf None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.199074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881100-6c54-404e-a855-94b9353460cf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.199417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7881100-6c54-404e-a855-94b9353460cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.251567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a7881100-6c54-404e-a855-94b9353460cf None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:18.260451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a7881100-6c54-404e-a855-94b9353460cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:18.335288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e04ab7-cb42-4c68-ba1c-65543bd67985 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.347624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.348334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.348618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.348894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.349602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.353844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.404207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 889/1783] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:18 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:18.408342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:18.418050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:18.443343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67860b7c-2173-4e4a-9531-6cc61852fb1d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.481869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.492115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.492115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.492115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.505423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.507247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.544843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 895/1784] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:18.563610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 890/1785] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:18.578261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881100-6c54-404e-a855-94b9353460cf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:18.596525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7881100-6c54-404e-a855-94b9353460cf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.619392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.620849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.621670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.621969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] PATH_INFO: `/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75/users/5169807d3f324ec19453a79eb1f6a287/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.652032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5dca307a4b443b9ed7d0ac61505d75, user_id=5169807d3f324ec19453a79eb1f6a287, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.654146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.685675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 896/1786] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:18 2022] PUT /identity/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75/users/5169807d3f324ec19453a79eb1f6a287/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:18.690033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:18.708549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c94e38-0f13-495b-985c-eb34ada59d01 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.832315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.833353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.833730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.834105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.835005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.836905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a639d0-d707-4f48-a334-d0223ac38ced admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.865476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:18.867196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 891/1787] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:18.877185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb9204-c1ac-465e-a414-42decc4f7221 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.877520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb9204-c1ac-465e-a414-42decc4f7221 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.877809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb9204-c1ac-465e-a414-42decc4f7221 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.879804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 892/1788] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 13:06:18.889422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4af191-e356-465c-9845-009e4f747c5a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:18.916392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:18.917240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.917363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.917451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.917527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.917731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] PATH_INFO: `/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75/users/5169807d3f324ec19453a79eb1f6a287/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.917839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:18.924944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5dca307a4b443b9ed7d0ac61505d75, user_id=5169807d3f324ec19453a79eb1f6a287, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:18.933752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d085fa-625d-4f8f-bd39-337500dbd045 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:18.973214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:18.976998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:18.980738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 897/1789] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:18 2022] PUT /identity/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75/users/5169807d3f324ec19453a79eb1f6a287/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:18.995840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:18.996147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:18.996432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:19.111139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] MFA Rules not processed for user `5169807d3f324ec19453a79eb1f6a287`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:19.112392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:19.189453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 893/1790] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:19.191733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:19.193998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:19.210281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3d436-cf5d-4b7f-ab8f-d7f7b5499950 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:19.217008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:19.217670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:19.217940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:19.218219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:19.228565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:19.230199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:19.231003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 898/1791] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:19.231857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:19.233543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:19.245575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 894/1792] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:19.246687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:19.248051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:19.249431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dc6e7-bf0f-46fa-a4fe-1c4eeb29078c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:19.255725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:19.256482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:19.256916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:19.257200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:19.264381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:19.267223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:19.269490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:19.284415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 899/1793] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:19.323248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ff35f-dcfb-4884-978b-08fd4aa1bb1d service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:19.328029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:19.329778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:19.330165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:19.330537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:19.345234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:19.349584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:19.362230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 895/1794] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:19.408596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:19.410127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:19.422663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a7c5-da78-46da-a4a0-08814cfcb2c9 service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:19.426833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:19.427500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:19.427785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:19.428060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:19.485471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:19.486971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141ac8f-4d9f-4c18-8534-eb7cbba514c6 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:19.497612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 900/1795] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:23.086036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:23.164685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbae8d-a3dd-4b24-82bb-159e416e2e06 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:23.169578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:23.170585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:23.170958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:23.171322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] PATH_INFO: `/v3/users/65c27fed42a046b1a47e4dd6f8007fe1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:23.172623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c27fed42a046b1a47e4dd6f8007fe1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:23.174822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:23.241720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c27fed42a046b1a47e4dd6f8007fe1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:23.252993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c27fed42a046b1a47e4dd6f8007fe1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:23.254016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c27fed42a046b1a47e4dd6f8007fe1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:23.278905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c27fed42a046b1a47e4dd6f8007fe1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:23.279813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c27fed42a046b1a47e4dd6f8007fe1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:23.312839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 896/1796] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:23 2022] DELETE /identity/v3/users/65c27fed42a046b1a47e4dd6f8007fe1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:23.318404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:23.319030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:23.319030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:23.341885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:23.343397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:23.370907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:23.431419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 901/1797] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:23.441263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:23.442760 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:23.513827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:23.540611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb1f7e-9976-41b2-8d99-fe5cc88d975e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:23.545103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:23.545886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:23.546735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:23.547134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:23.570517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:23.572388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9738e-1f3f-44dd-b348-80b3445060ee service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:23.586988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 897/1798] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:23.915004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:23.917610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:23.935847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff46d8-5d57-4240-a349-b7755ffdfe22 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:23.951398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:23.952745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:23.953109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:23.953500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] PATH_INFO: `/v3/users/dea452c43c274562a23b6621870d8631` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:23.955303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea452c43c274562a23b6621870d8631)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:23.958342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:24.043157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea452c43c274562a23b6621870d8631'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.054532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea452c43c274562a23b6621870d8631'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.055367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea452c43c274562a23b6621870d8631'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.080887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea452c43c274562a23b6621870d8631'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.082045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea452c43c274562a23b6621870d8631'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.110392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 902/1799] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:23 2022] DELETE /identity/v3/users/dea452c43c274562a23b6621870d8631 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:24.115543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:24.116092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:24.116092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:24.117543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:24.119419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.154961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:24.156649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.203975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eda982-a1ca-475b-8347-b74e7a67c126 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:24.209005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:24.209886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:24.210193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:24.210503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] PATH_INFO: `/v3/projects/383fccfcf18341cbbaea31a5154cdfb1` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:24.211744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=383fccfcf18341cbbaea31a5154cdfb1)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:24.213670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:24.224130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 898/1800] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:24.233820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:24.235297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.254804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:24.267794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383fccfcf18341cbbaea31a5154cdfb1', 'request_id': 'req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.276689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383fccfcf18341cbbaea31a5154cdfb1', 'request_id': 'req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.283199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383fccfcf18341cbbaea31a5154cdfb1', 'request_id': 'req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.294297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] The token cache is being invalidate because project 383fccfcf18341cbbaea31a5154cdfb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:24.294620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.295267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.296852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 903/1801] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:24 2022] DELETE /identity/v3/projects/383fccfcf18341cbbaea31a5154cdfb1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:24.301041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:24.301240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a309e5-bf5a-4fb0-93bb-956ddb53c115 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:24.302571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.358275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:24.358947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:24.359229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:24.359563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:24.361297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.379201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee97b5-b7c7-4746-b608-fc4ba12ab09f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:24.387359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:24.387995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:24.388131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:24.388386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] PATH_INFO: `/v3/users/5169807d3f324ec19453a79eb1f6a287` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:24.391723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5169807d3f324ec19453a79eb1f6a287)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:24.393655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:24.418609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:24.443015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:24.447048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:24.461748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 899/1802] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:24.482165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5169807d3f324ec19453a79eb1f6a287'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.492908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5169807d3f324ec19453a79eb1f6a287'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.493875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5169807d3f324ec19453a79eb1f6a287'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.511553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5169807d3f324ec19453a79eb1f6a287'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.512492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5169807d3f324ec19453a79eb1f6a287'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.527039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 904/1803] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:24 2022] DELETE /identity/v3/users/5169807d3f324ec19453a79eb1f6a287 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:24.765663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:24.767508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.789346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b0134-89d5-4ca1-82c7-3d5f6d21ac3f service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:24.876834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:24.877767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:24.878022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:24.878311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] PATH_INFO: `/v3/users/3cb4bcc4e9c64406a8d7b13151bc05e8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:24.880261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb4bcc4e9c64406a8d7b13151bc05e8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:24.882501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:24.974182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:24.975856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:24.979540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb4bcc4e9c64406a8d7b13151bc05e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.996240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb4bcc4e9c64406a8d7b13151bc05e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:24.999129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb4bcc4e9c64406a8d7b13151bc05e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.004836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad11e2-0972-41a6-8a4a-a8b39ff16734 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.012416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.020889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb4bcc4e9c64406a8d7b13151bc05e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.021677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb4bcc4e9c64406a8d7b13151bc05e8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.027239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.028631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.028921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] PATH_INFO: `/v3/projects/c6acab5e8aaf4c4e9a9d5a3971256222` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.031653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6acab5e8aaf4c4e9a9d5a3971256222)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.035431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 900/1804] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:24 2022] DELETE /identity/v3/users/3cb4bcc4e9c64406a8d7b13151bc05e8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:25.036330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.037697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.039321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.096211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:25.113252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6acab5e8aaf4c4e9a9d5a3971256222', 'request_id': 'req-f9cf60a9-9fce-4899-b764-70cffc8e606e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.123942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb820f53-4070-4b76-8020-9266fab6d9e8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.125833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6acab5e8aaf4c4e9a9d5a3971256222', 'request_id': 'req-f9cf60a9-9fce-4899-b764-70cffc8e606e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.135419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:25.135898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6acab5e8aaf4c4e9a9d5a3971256222', 'request_id': 'req-f9cf60a9-9fce-4899-b764-70cffc8e606e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.151313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] The token cache is being invalidate because project c6acab5e8aaf4c4e9a9d5a3971256222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:25.151718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.152413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f9cf60a9-9fce-4899-b764-70cffc8e606e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.156571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 905/1805] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:24 2022] DELETE /identity/v3/projects/c6acab5e8aaf4c4e9a9d5a3971256222 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:25.158662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.158662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.159156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.185405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:25.186695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.219357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.220563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.220831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.221158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] PATH_INFO: `/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.222730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5dca307a4b443b9ed7d0ac61505d75)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.224939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.283317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 906/1806] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:25.284897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.287466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.310037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc62c5-70c1-44af-bbdc-80327a4c4d42 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.319266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5dca307a4b443b9ed7d0ac61505d75', 'request_id': 'req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.334700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5dca307a4b443b9ed7d0ac61505d75', 'request_id': 'req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.346756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5dca307a4b443b9ed7d0ac61505d75', 'request_id': 'req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.367970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] The token cache is being invalidate because project 6d5dca307a4b443b9ed7d0ac61505d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:25.368322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.369205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.371121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 901/1807] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:25 2022] DELETE /identity/v3/projects/6d5dca307a4b443b9ed7d0ac61505d75 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:25.379630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.381321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.395433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.396638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.396922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.397194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] PATH_INFO: `/v3/users/c2fdb9e88ecb401d989c692521fd8e05` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.398636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fdb9e88ecb401d989c692521fd8e05)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.406384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.462312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6e8c8-68ba-40d4-b7af-14e39a1faf26 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.468394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.469099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.469377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.469650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.472178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.488651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fdb9e88ecb401d989c692521fd8e05'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.490587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.492057 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f196d8b-1e52-4434-999a-63698d1a675a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.500540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fdb9e88ecb401d989c692521fd8e05'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.501311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fdb9e88ecb401d989c692521fd8e05'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.503646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 902/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:25.505053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.505339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.505613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.515903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:25.518833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fdb9e88ecb401d989c692521fd8e05'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.520097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-84767519-14dc-476f-9ca1-41f96e3dd4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fdb9e88ecb401d989c692521fd8e05'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:25.525338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:25.527054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.542196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 907/1809] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:25 2022] DELETE /identity/v3/users/c2fdb9e88ecb401d989c692521fd8e05 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:25.542473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 903/1810] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:25.547600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.548003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.548486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.579020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:25.581254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:25.667958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 908/1811] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:25.672510 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.684046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334d34f-9466-4fe4-967b-25fbe659ef68 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.689189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.689967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.690313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.690684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.691766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:25.692143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.693856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.711397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 904/1812] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:25.712674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.724218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1937fc-fb06-4bd9-a37c-580bc6935d44 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.730659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.731418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.732192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.732509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.733320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.735374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.772123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 909/1813] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:25 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:25.773774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.786158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66007b-f78f-4412-80e7-1a03299b9b18 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.822810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:25.869208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.870729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.871013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.871292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.872006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.874084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.900086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 905/1814] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:25 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:25.904544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.915880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33a4d9-ace2-4503-9665-f8a181b23b87 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.921136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.921970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.922364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.922687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.923536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.925242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.942965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 910/1815] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:25.944690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.958023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d2bb8-e15a-4b79-acd6-2670fd622240 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:25.965173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:25.965860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:25.966140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:25.966515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:25.967909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:25.970555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc52d9-3309-4679-9851-b23501c814af admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:25.989632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:25.989935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 906/1816] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:26.004652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04e1ce-a527-4b7e-877f-cbb972c032fb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.015023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.016268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.016555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.016834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] PATH_INFO: `/v3/projects/da78a05d8d584ee6aae8388ea25de3cb/users/e990627c51f9499092041090cbef937c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.057043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da78a05d8d584ee6aae8388ea25de3cb, user_id=e990627c51f9499092041090cbef937c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.060829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.094952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.095247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.095499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.097593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 911/1817] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:25 2022] PUT /identity/v3/projects/da78a05d8d584ee6aae8388ea25de3cb/users/e990627c51f9499092041090cbef937c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:26.148428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] MFA Rules not processed for user `e990627c51f9499092041090cbef937c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:26.151842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:26.275523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.278284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 907/1818] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:26.355923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95211a-f77e-45fd-be10-d2d551f8557e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.361927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.362857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.363232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.363578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.364493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.367745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.412051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 912/1819] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:26 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:26.425180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.443784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb3185-dcb1-46fa-bc55-d0338b01b10f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.598351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.598804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.599140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.599576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.600464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.602950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.634371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 908/1820] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:26 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:26.638000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.651010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef0440-c733-41c9-8e5e-38c94291843a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.659293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.659735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.660664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.661099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.662741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.665634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.680055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 913/1821] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:26.685415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.698640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6f593-20c4-4d32-ab6c-b545300cb1b1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.708352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.709523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.709972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.711397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] PATH_INFO: `/v3/projects/2086bfd777514194b47bc152e6c7fb6e/users/815eef8a281f4a4aba119225fbec6dc7/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.743584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2086bfd777514194b47bc152e6c7fb6e, user_id=815eef8a281f4a4aba119225fbec6dc7, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.748248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.778353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:26.793605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 909/1822] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:26 2022] PUT /identity/v3/projects/2086bfd777514194b47bc152e6c7fb6e/users/815eef8a281f4a4aba119225fbec6dc7/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:26.800337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.813799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:26.844308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.846441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9149c51-d65c-4c95-be40-ebdb45878c45 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:26.847253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:26.952179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:26.953062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:26.953493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:26.953726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:26.955364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:26.957443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:26.977434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 914/1823] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:26.979250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:26.983980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:26.990533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71302b-7152-4292-8609-47ac23314b9c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.000251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.001342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da1e39-314d-41af-af53-287111d48895 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.001485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.001842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.002185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] PATH_INFO: `/v3/projects/0715fbf81eb54d998ef113856fcc317f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.005894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0715fbf81eb54d998ef113856fcc317f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.008960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.010226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.010761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.010990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.011568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] PATH_INFO: `/v3/projects/2db2618a83884e7bbe98af42294212fe` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.012910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db2618a83884e7bbe98af42294212fe)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.016392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.068478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0715fbf81eb54d998ef113856fcc317f', 'request_id': 'req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.087623 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0715fbf81eb54d998ef113856fcc317f', 'request_id': 'req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.096658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0715fbf81eb54d998ef113856fcc317f', 'request_id': 'req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.103704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db2618a83884e7bbe98af42294212fe', 'request_id': 'req-16a6a3d5-6137-40f0-80ea-41857d2bc696'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.127161 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db2618a83884e7bbe98af42294212fe', 'request_id': 'req-16a6a3d5-6137-40f0-80ea-41857d2bc696'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.127921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] The token cache is being invalidate because project 0715fbf81eb54d998ef113856fcc317f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:27.128198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.130126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.142789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 910/1824] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:26 2022] DELETE /identity/v3/projects/0715fbf81eb54d998ef113856fcc317f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:27.143874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.148199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.154488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db2618a83884e7bbe98af42294212fe', 'request_id': 'req-16a6a3d5-6137-40f0-80ea-41857d2bc696'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.166679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] The token cache is being invalidate because project 2db2618a83884e7bbe98af42294212fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:27.166924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.167664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.173194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 915/1825] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:26 2022] DELETE /identity/v3/projects/2db2618a83884e7bbe98af42294212fe => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:27.174276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.175842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.252378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1370b2-0e46-43e8-aac1-52205cf7d3d3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.258073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.259284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.259660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.260032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] PATH_INFO: `/v3/projects/2086bfd777514194b47bc152e6c7fb6e/users/815eef8a281f4a4aba119225fbec6dc7/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.260495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6a3d5-6137-40f0-80ea-41857d2bc696 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.265840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2086bfd777514194b47bc152e6c7fb6e, user_id=815eef8a281f4a4aba119225fbec6dc7, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.269735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c796d4b-5448-49d4-83fc-49119bcbe8da admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.274460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.275472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.275766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.276047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] PATH_INFO: `/v3/users/ea7c6f0ebff946f69438ee280c613ff8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.277364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7c6f0ebff946f69438ee280c613ff8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.279608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.286157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 911/1826] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:27 2022] PUT /identity/v3/projects/2086bfd777514194b47bc152e6c7fb6e/users/815eef8a281f4a4aba119225fbec6dc7/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:27.287617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49be0-f7e1-4ffa-ad7a-1d3dc0864181 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.287904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49be0-f7e1-4ffa-ad7a-1d3dc0864181 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.288279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49be0-f7e1-4ffa-ad7a-1d3dc0864181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.308421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-90a49be0-f7e1-4ffa-ad7a-1d3dc0864181 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:27.311056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-90a49be0-f7e1-4ffa-ad7a-1d3dc0864181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.323523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:27.334535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:27.361808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7c6f0ebff946f69438ee280c613ff8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.378082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7c6f0ebff946f69438ee280c613ff8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.379903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7c6f0ebff946f69438ee280c613ff8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.391864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 912/1827] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:27.392976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7c6f0ebff946f69438ee280c613ff8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.393455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.393694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-bfa237c7-4f94-4714-bc33-29497672eaaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7c6f0ebff946f69438ee280c613ff8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.393798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.393996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.407259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 916/1828] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:27 2022] DELETE /identity/v3/users/ea7c6f0ebff946f69438ee280c613ff8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:27.408539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.408827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.409101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.429256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:27.431445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.433743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] MFA Rules not processed for user `815eef8a281f4a4aba119225fbec6dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:27.434965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.507633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 917/1829] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:27.511708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.522480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96179e-41ad-4963-a735-7c1622098855 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.532294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.532981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.533667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.533667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.534259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:27.535112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.538493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.545388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 913/1830] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:27.558378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.561404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.569885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 918/1831] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:27.583540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.596085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9af25-9876-45a9-8786-9a992ac7327d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.601981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.606701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.607088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.607370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.608073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.610959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.654493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 919/1832] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:27 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:27.655746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.678296 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377241da-16f5-4f5a-9b8e-591127438aec None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.754499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.755246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.756538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.756895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.763806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47511572-c8a9-493a-818f-a7ce7b7e3002 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.768841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.769593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.769842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.770116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.771373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.773591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.780884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:27.786112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:27.809283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 920/1833] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:27 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:27.810103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.813441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.833579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2657876-2b6f-41bd-a069-e1521416c4f6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.838878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.841345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.841345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.841345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] PATH_INFO: `/v3/users/3f4a86becf594f00a2bbdcf3a23056b6` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.842850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4a86becf594f00a2bbdcf3a23056b6)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.844847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.848062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.850192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.863449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 914/1834] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:27.865170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.879395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded125e6-24b3-4069-a9ee-359db40183c9 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.885048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.886564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.886852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.887129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.888179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.890149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.902391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 915/1835] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:27.903376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.905138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:27.920451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422d45a-9526-4df8-8b82-00a59c4b882d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:27.927210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:27.928181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:27.928517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:27.928734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] PATH_INFO: `/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:27.929936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae8726c4cca403dbbf4f61ec118cdb0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:27.931951 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:27.943793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4a86becf594f00a2bbdcf3a23056b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.960368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4a86becf594f00a2bbdcf3a23056b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.961882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4a86becf594f00a2bbdcf3a23056b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.978280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4a86becf594f00a2bbdcf3a23056b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.979070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4a86becf594f00a2bbdcf3a23056b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.993335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae8726c4cca403dbbf4f61ec118cdb0', 'request_id': 'req-abebe088-ad36-4e01-9f59-17f3508fa02a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:27.993459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 921/1836] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:27 2022] DELETE /identity/v3/users/3f4a86becf594f00a2bbdcf3a23056b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:27.994084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:27.996616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.007694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae8726c4cca403dbbf4f61ec118cdb0', 'request_id': 'req-abebe088-ad36-4e01-9f59-17f3508fa02a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.019353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae8726c4cca403dbbf4f61ec118cdb0', 'request_id': 'req-abebe088-ad36-4e01-9f59-17f3508fa02a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.031242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] The token cache is being invalidate because project 1ae8726c4cca403dbbf4f61ec118cdb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:28.031514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.032758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.034849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 916/1837] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:27 2022] DELETE /identity/v3/projects/1ae8726c4cca403dbbf4f61ec118cdb0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:28.035576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.036994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.081650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c26222-6c19-45e3-b0c2-6efec4da3f85 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.086456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.087212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.087501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.087775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.088515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.090506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5189273-6127-4af7-99e3-7de86e885f84 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.100420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 922/1838] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:28.101622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76745507-a969-41ce-b935-f8331ae579b6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.101907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76745507-a969-41ce-b935-f8331ae579b6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.102183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76745507-a969-41ce-b935-f8331ae579b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.103211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe088-ad36-4e01-9f59-17f3508fa02a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.107904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.108839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.109187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.109529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] PATH_INFO: `/v3/users/e990627c51f9499092041090cbef937c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.110838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e990627c51f9499092041090cbef937c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.112942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.119905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-76745507-a969-41ce-b935-f8331ae579b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:28.121130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-76745507-a969-41ce-b935-f8331ae579b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.128936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 923/1839] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:28.130166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.130495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.130774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.148935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:28.149416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:28.150801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.159223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 924/1840] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:28.160065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.161738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.173329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9941d8d3-7cbe-4e1e-84d4-42ee81370db5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.173455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e990627c51f9499092041090cbef937c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.178844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.179835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.180044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.180350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] PATH_INFO: `/v3/projects/79d771fb5b76472eb4324a2a7ddf92ed/users/60a7f9559eac4a91a890bf8bbf80a85c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.184998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e990627c51f9499092041090cbef937c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.186116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e990627c51f9499092041090cbef937c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.196585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d771fb5b76472eb4324a2a7ddf92ed, user_id=60a7f9559eac4a91a890bf8bbf80a85c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.199473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.215756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e990627c51f9499092041090cbef937c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.216463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e24a71a8-ba88-4eab-b6a7-43d02b17b173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e990627c51f9499092041090cbef937c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:28.218373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 925/1841] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:28 2022] PUT /identity/v3/projects/79d771fb5b76472eb4324a2a7ddf92ed/users/60a7f9559eac4a91a890bf8bbf80a85c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:28.218503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.220119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.233441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 917/1842] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:28 2022] DELETE /identity/v3/users/e990627c51f9499092041090cbef937c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:28.235114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36906be-4448-4391-9823-22ac3603d433 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.235422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36906be-4448-4391-9823-22ac3603d433 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.235699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36906be-4448-4391-9823-22ac3603d433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.260346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b36906be-4448-4391-9823-22ac3603d433 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:28.261694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b36906be-4448-4391-9823-22ac3603d433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.264738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:28.291934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b0068-0e3f-460b-84ac-dc2902e71ee5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.348323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.348977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.349258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.349463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 918/1843] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:28.349641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.350800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.351070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.351346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.351670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.364787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.366067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad5aae-e212-464a-b0cc-79842f4f3644 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.381064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 926/1844] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:28.382607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] MFA Rules not processed for user `60a7f9559eac4a91a890bf8bbf80a85c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:28.383871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.392563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.392932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.393279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.416918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:28.419007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.429359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 927/1845] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:28.430347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.443921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f71c50-685d-4982-8c3e-0b287014663c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.450970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.451857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.452171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.452610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.453527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:28.454011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.456397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.463406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 919/1846] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:28.464900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.478064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3dbcd9-f976-46a7-9511-4480e1c7a3f7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.482033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 928/1847] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:28.482033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.484208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.486286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.487136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.487555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.488081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.498924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.500614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.500759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796dc33-989c-4804-85fd-54033eefb9a5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.508908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.509665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.510047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.510485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.512226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.512344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 920/1848] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:28.513199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.514397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.527132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82caceb2-f5df-4448-a599-d71b9dbaa4bf service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.532915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.533852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.534295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.534878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.535762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.537890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.551339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 929/1849] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:28.555529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.574430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:28.594670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 921/1850] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:28.599313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.647414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766fc69-7344-4d5f-9bbe-a8e49a8822e2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.695792 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466097f-b59f-4a19-9cb6-3059d3b44246 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.704752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.709407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.709759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.710110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.711083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.713121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.736403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:28.776333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.777407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.778106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.778252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.780352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.783598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.786866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 922/1851] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:28.790756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.807288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe093a3f-452f-428b-9c29-fc3d6f8ac26f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.814683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.815706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.816054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.816428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.817441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.819775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.860135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 923/1852] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:28.861009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.874488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:28.882634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 930/1853] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:28 2022] POST /identity/v3/users => generated 300 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:28.884456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.896185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd444-3bfc-4c1f-80b7-1fa03ea78061 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.901645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.902346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.902652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.902925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.903657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.905263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.914734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ea562-6cf9-462b-9e66-c821d01b545d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.922652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.923373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 931/1854] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:28.924386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.925145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.926032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.926360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] PATH_INFO: `/v3/projects/ac3bd7ca36cd417ab6e235555d9a85f9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.927943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3bd7ca36cd417ab6e235555d9a85f9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.930530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.942993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2368-0114-4cd3-8ece-e73a774c4e7f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.949296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.950271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.950805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.951334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:28.952293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:28.954493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:28.969809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 932/1855] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:28.970915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:28.984482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa966f1-6623-4bdd-90c1-0665fed522f9 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:28.995108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:28.997159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:28.998592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:28.998912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] PATH_INFO: `/v3/projects/d9f94c50ecf941e2999f231bce522081/users/8a9498858dfa4015933f9545afddd148/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.016588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f94c50ecf941e2999f231bce522081, user_id=8a9498858dfa4015933f9545afddd148, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.020672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.047679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3bd7ca36cd417ab6e235555d9a85f9', 'request_id': 'req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.059142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 933/1856] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:28 2022] PUT /identity/v3/projects/d9f94c50ecf941e2999f231bce522081/users/8a9498858dfa4015933f9545afddd148/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:29.059142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.066855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3bd7ca36cd417ab6e235555d9a85f9', 'request_id': 'req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.088522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3bd7ca36cd417ab6e235555d9a85f9', 'request_id': 'req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.118100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] The token cache is being invalidate because project ac3bd7ca36cd417ab6e235555d9a85f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:29.118436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.127904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc653f2a-b63d-48ad-8e6e-97bf4147cb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.133288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 924/1857] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:28 2022] DELETE /identity/v3/projects/ac3bd7ca36cd417ab6e235555d9a85f9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:29.139569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.141272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.141626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.175697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82ccaa-e03e-44a6-9acf-fa9e4fc9b45f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.183391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] MFA Rules not processed for user `8a9498858dfa4015933f9545afddd148`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:29.184738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.186257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.187207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.187492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.187768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] PATH_INFO: `/v3/projects/7bf8ac2883264666805847f650fe2c90/users/a434de79e074483291f8689f729a22f4/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.215216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf8ac2883264666805847f650fe2c90, user_id=a434de79e074483291f8689f729a22f4, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.218398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.242355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 934/1858] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:29 2022] PUT /identity/v3/projects/7bf8ac2883264666805847f650fe2c90/users/a434de79e074483291f8689f729a22f4/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:29.246433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.248210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:29.249429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.305639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 925/1859] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:29.307748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.309989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.329814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211935e-c111-4ff3-a049-cc244ce19317 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.338001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e26d6d-e9c9-44e6-b6f4-92ad2024cb14 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.342134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.344088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.344970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.345257 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.345533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] PATH_INFO: `/v3/users/c8b2a7c4e6f44f9b998dd1c800de2871` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.345752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.347904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b2a7c4e6f44f9b998dd1c800de2871)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.348278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.348868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.349143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.350279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.352590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.367640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 926/1860] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:29.368952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.372391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.381526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:29.399823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbd2b6-8aa8-4e36-9a4f-6ec9a83973aa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.425467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.426175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.429185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.429524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.430278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.436639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.461888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b2a7c4e6f44f9b998dd1c800de2871'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.481762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 927/1861] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:29 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:29.483575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.490765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b2a7c4e6f44f9b998dd1c800de2871'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.492238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b2a7c4e6f44f9b998dd1c800de2871'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.518833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b2a7c4e6f44f9b998dd1c800de2871'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.520210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b2a7c4e6f44f9b998dd1c800de2871'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:29.566366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 935/1862] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:29 2022] DELETE /identity/v3/users/c8b2a7c4e6f44f9b998dd1c800de2871 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:29.567314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.568753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.627171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa14a4-3f1b-4359-b53a-323f9d141501 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.702001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14dce38-a8cd-4df6-a32c-a3a5d21c19b6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.708457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.709380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.709668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.709943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.710738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.712848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.767643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 936/1863] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:29 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:29.768526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.769576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.770377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.771085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.771365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] PATH_INFO: `/v3/projects/7bf8ac2883264666805847f650fe2c90/users/a434de79e074483291f8689f729a22f4/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.777389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf8ac2883264666805847f650fe2c90, user_id=a434de79e074483291f8689f729a22f4, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.784424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b2b8e-f687-4063-a864-bf3f47ec0038 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.797576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78247d4-4875-4df4-a1c9-5bcd1c8cf0a0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.825826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 928/1864] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:29 2022] PUT /identity/v3/projects/7bf8ac2883264666805847f650fe2c90/users/a434de79e074483291f8689f729a22f4/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:29.838397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7684116-7899-4208-8726-092939eb52f2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.838841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7684116-7899-4208-8726-092939eb52f2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.839272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7684116-7899-4208-8726-092939eb52f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.892323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d7684116-7899-4208-8726-092939eb52f2 None None] MFA Rules not processed for user `a434de79e074483291f8689f729a22f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:29.893789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d7684116-7899-4208-8726-092939eb52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:29.920332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.921513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.921794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.922066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:29.922873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:29.925630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:29.944032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 937/1865] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:29.949325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:29.965013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd015621-36b9-4677-a7b7-b8de45f420b4 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:29.966781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:29.974269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:29.975456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:29.976057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:29.977354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] PATH_INFO: `/v3/projects/fe6b24733f7244afa86ae30de81a45df/users/f84291363651455985c8b65e5e3375d9/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.014739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6b24733f7244afa86ae30de81a45df, user_id=f84291363651455985c8b65e5e3375d9, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.016175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 929/1866] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:30.017221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7684116-7899-4208-8726-092939eb52f2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.018784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d7684116-7899-4208-8726-092939eb52f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.019855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.036777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:30.041222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7684116-7899-4208-8726-092939eb52f2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.055642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 938/1867] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:29 2022] PUT /identity/v3/projects/fe6b24733f7244afa86ae30de81a45df/users/f84291363651455985c8b65e5e3375d9/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:30.059804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.063138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.175783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87912f-4f71-45af-94ca-434287929166 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.185866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.186878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.188149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.188534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.202537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.203571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.204235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.206542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] PATH_INFO: `/v3/projects/da78a05d8d584ee6aae8388ea25de3cb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.210076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=da78a05d8d584ee6aae8388ea25de3cb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.214350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.324333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da78a05d8d584ee6aae8388ea25de3cb', 'request_id': 'req-9debeed3-dc68-4da5-8911-453f4b88f1db'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.336039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da78a05d8d584ee6aae8388ea25de3cb', 'request_id': 'req-9debeed3-dc68-4da5-8911-453f4b88f1db'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.342464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.343938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.346880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da78a05d8d584ee6aae8388ea25de3cb', 'request_id': 'req-9debeed3-dc68-4da5-8911-453f4b88f1db'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.364126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] The token cache is being invalidate because project da78a05d8d584ee6aae8388ea25de3cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:30.365529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.366688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.366838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 939/1868] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:30.367704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.369290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 930/1869] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:30 2022] DELETE /identity/v3/projects/da78a05d8d584ee6aae8388ea25de3cb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:30.373105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.375568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.388958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8afd0a-d1df-4e1b-ba10-71e2b8261c57 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.493901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.494634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.494923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.495199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.495947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.500468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.517292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debeed3-dc68-4da5-8911-453f4b88f1db admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.518311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 940/1870] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:30.523208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.525570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.544824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.545774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.546649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.548044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] PATH_INFO: `/v3/users/815eef8a281f4a4aba119225fbec6dc7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.549076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=815eef8a281f4a4aba119225fbec6dc7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.553412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.563221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfff3c3-e5bb-4d51-8451-27e3c857bdf5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.584890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.586892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.587183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.587466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] PATH_INFO: `/v3/projects/fe6b24733f7244afa86ae30de81a45df/users/f84291363651455985c8b65e5e3375d9/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.595895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6b24733f7244afa86ae30de81a45df, user_id=f84291363651455985c8b65e5e3375d9, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.599380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.617803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 941/1871] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:30 2022] PUT /identity/v3/projects/fe6b24733f7244afa86ae30de81a45df/users/f84291363651455985c8b65e5e3375d9/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:30.618652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.620381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.635099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:30.679718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:30.697772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815eef8a281f4a4aba119225fbec6dc7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.709822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815eef8a281f4a4aba119225fbec6dc7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.711033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815eef8a281f4a4aba119225fbec6dc7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.720364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de2f22-a79c-4255-b72a-c53995328276 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.725625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.726579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.726869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.727144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] PATH_INFO: `/v3/users/60a7f9559eac4a91a890bf8bbf80a85c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.727753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815eef8a281f4a4aba119225fbec6dc7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.728439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a7f9559eac4a91a890bf8bbf80a85c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.729035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b352c13c-c392-4456-b787-96d496939983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815eef8a281f4a4aba119225fbec6dc7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.730542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:30.745612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 931/1872] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:30 2022] DELETE /identity/v3/users/815eef8a281f4a4aba119225fbec6dc7 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:30.748591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.748591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.748591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.789662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] MFA Rules not processed for user `f84291363651455985c8b65e5e3375d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:30.791772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.832467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a7f9559eac4a91a890bf8bbf80a85c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.843565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a7f9559eac4a91a890bf8bbf80a85c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.844515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a7f9559eac4a91a890bf8bbf80a85c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.861037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a7f9559eac4a91a890bf8bbf80a85c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.862003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-69dd2f47-17b9-417b-b5a3-11f12342885b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a7f9559eac4a91a890bf8bbf80a85c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:30.881450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 932/1873] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:30.882577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.887384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.887543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 942/1874] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:30 2022] DELETE /identity/v3/users/60a7f9559eac4a91a890bf8bbf80a85c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:30.889922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677614d8-b511-4a76-975a-a50494c92915 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.890285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677614d8-b511-4a76-975a-a50494c92915 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.890659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677614d8-b511-4a76-975a-a50494c92915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.910368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-677614d8-b511-4a76-975a-a50494c92915 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:30.911654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-677614d8-b511-4a76-975a-a50494c92915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.977849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d001911-9cd0-427d-8bb3-dd4093acf225 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:30.982334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 943/1875] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:30.983396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677614d8-b511-4a76-975a-a50494c92915 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:30.985309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-677614d8-b511-4a76-975a-a50494c92915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:30.986613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:30.987234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:30.988014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:30.988014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] PATH_INFO: `/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:30.989201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee53f071ca148eda21ac90aaf3d7f32)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:30.991214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.021013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:31.053550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee53f071ca148eda21ac90aaf3d7f32', 'request_id': 'req-e383155e-9969-4309-99de-a74d562b0e68'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.060805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677614d8-b511-4a76-975a-a50494c92915 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.064335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee53f071ca148eda21ac90aaf3d7f32', 'request_id': 'req-e383155e-9969-4309-99de-a74d562b0e68'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.064788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:31.072995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee53f071ca148eda21ac90aaf3d7f32', 'request_id': 'req-e383155e-9969-4309-99de-a74d562b0e68'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.084542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] The token cache is being invalidate because project 7ee53f071ca148eda21ac90aaf3d7f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:31.084908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.086307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.087662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 933/1876] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:30 2022] DELETE /identity/v3/projects/7ee53f071ca148eda21ac90aaf3d7f32 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:31.088576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.090275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.121151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.122045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.122285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.122689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.124699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.176681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e383155e-9969-4309-99de-a74d562b0e68 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.180747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.181404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.181689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.181962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.184068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.207113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.208590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df2cc9-251a-4d2c-8e21-74912bd6face service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.220053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 944/1877] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:31.221425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6f54-cef3-4a96-8381-c67dbbb43512 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.221715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6f54-cef3-4a96-8381-c67dbbb43512 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.222147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da6f54-cef3-4a96-8381-c67dbbb43512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.245113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d3da6f54-cef3-4a96-8381-c67dbbb43512 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:31.246767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d3da6f54-cef3-4a96-8381-c67dbbb43512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.263781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.265573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.277004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 934/1878] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:31.277880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.279475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.296415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf03ca-711b-466f-b8b7-c7d975ff6e95 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.318876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.319948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.320300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.320874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] PATH_INFO: `/v3/projects/2086bfd777514194b47bc152e6c7fb6e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.322746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2086bfd777514194b47bc152e6c7fb6e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.326359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 945/1879] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:31.326504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.332760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.334612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.335052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.358550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:31.360228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.375298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 946/1880] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:31.383049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.387400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2086bfd777514194b47bc152e6c7fb6e', 'request_id': 'req-00d59650-8da7-4da7-9a08-0a035f9295d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.393186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:31.417669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a69699-ff0b-4de5-a457-d9aa02bb61c7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.438259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2086bfd777514194b47bc152e6c7fb6e', 'request_id': 'req-00d59650-8da7-4da7-9a08-0a035f9295d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.447007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2086bfd777514194b47bc152e6c7fb6e', 'request_id': 'req-00d59650-8da7-4da7-9a08-0a035f9295d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.474431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] The token cache is being invalidate because project 2086bfd777514194b47bc152e6c7fb6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:31.474995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.476794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-00d59650-8da7-4da7-9a08-0a035f9295d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:31.478034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:31.485829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 935/1881] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:31 2022] DELETE /identity/v3/projects/2086bfd777514194b47bc152e6c7fb6e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:31.488518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cee4b-b4d1-4b62-a0d0-07f29515256c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.488853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cee4b-b4d1-4b62-a0d0-07f29515256c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.489134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cee4b-b4d1-4b62-a0d0-07f29515256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.501480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.502205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.502564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.503584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.504395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:31.506028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.507985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.530673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b33cee4b-b4d1-4b62-a0d0-07f29515256c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:31.532055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b33cee4b-b4d1-4b62-a0d0-07f29515256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.535425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 947/1882] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:31.540041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.544252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.550038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 936/1883] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:31.558354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.558647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.558924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.576633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f195c-337e-49da-ac93-1aa24fe64a4b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.583509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:31.584961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.592106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.592792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.593073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.593352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.594064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.596157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 937/1884] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:31.599221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.600389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.611900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9facc-48c5-4df1-8bcc-f909ab89abe8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.618236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.619326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.619617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.619899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.620737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:31.621049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.623218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.639152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 948/1885] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:31.646146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.664830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 938/1886] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:31.671033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.686232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b8598-40d7-46f4-92e9-844627954089 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.771646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23baa025-e679-4c9b-ab4e-966dba63f1a5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.772602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.773339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.773625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.773899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.774640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.776747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.776858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.777535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.777949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.778296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.779242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.781229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02ff7e-de26-4078-9468-11fe81c35b0f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.811354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 949/1887] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:31.811354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.811354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.811354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.814971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 950/1888] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:06:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:06:31.815869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.817953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:31.821115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 939/1889] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:31.822461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.934592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e911a4-f3bc-4376-bc09-7297a6928315 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.937180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f15b3-a420-474c-a6b7-5c127c12717a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.939503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.940178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.940458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.940732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.941607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.942557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84291363651455985c8b65e5e3375d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe6b24733f7244afa86ae30de81a45df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.943701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.943832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.943911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.943911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.944725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:31.945433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.946705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11af7d5-9788-48aa-a98c-59605d702ccc tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:31.958207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 940/1890] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:31.959139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:31.960849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 951/1891] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:31.962044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.962351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.962650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.972917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8842bd-caeb-4243-936e-62130e488c2e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:31.980590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:31.981356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:31.981638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:31.982024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:31.983515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:31.985586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.002040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:32.003449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.011305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 941/1892] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:32.012806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.014984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.028355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d12252-4dc1-446e-a158-58cb02cb29b8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.038565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.040834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.040834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.040834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] PATH_INFO: `/v3/projects/79d771fb5b76472eb4324a2a7ddf92ed` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.041913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d771fb5b76472eb4324a2a7ddf92ed)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.044277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.056198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.064412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.091967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 952/1893] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:32.092908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.111618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d452e-287a-4753-bb10-50160e0064bb None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.115336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d771fb5b76472eb4324a2a7ddf92ed', 'request_id': 'req-43993567-81db-478d-a7cf-0425a757afed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.125090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d771fb5b76472eb4324a2a7ddf92ed', 'request_id': 'req-43993567-81db-478d-a7cf-0425a757afed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.133213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d771fb5b76472eb4324a2a7ddf92ed', 'request_id': 'req-43993567-81db-478d-a7cf-0425a757afed'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.148508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] The token cache is being invalidate because project 79d771fb5b76472eb4324a2a7ddf92ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:32.149012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.149685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.151711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 942/1894] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:32 2022] DELETE /identity/v3/projects/79d771fb5b76472eb4324a2a7ddf92ed => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:32.152220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.153542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.180530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.181212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.181619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.181913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.182680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.184273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.195308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 953/1895] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:32.196213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.198748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.211193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95590cb-a524-479b-adf0-18ea7731c716 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.216283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.216943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.217229 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.217503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.218286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.220349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.224094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993567-81db-478d-a7cf-0425a757afed admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.228711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84291363651455985c8b65e5e3375d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe6b24733f7244afa86ae30de81a45df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.229371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.229655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.229932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.230503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 954/1896] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:32.230676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.231300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.232581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.233034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.266041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 943/1897] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:32 2022] POST /identity/v3/projects => generated 354 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:32.266946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.268559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.317244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23f3d2-edea-44b3-8abc-4f6b11c67bad admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.334747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050436a6-fe07-4b6a-b813-a79667e44f5f tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.340031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.340912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.341191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.341462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] PATH_INFO: `/v3/users/a434de79e074483291f8689f729a22f4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.342682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a434de79e074483291f8689f729a22f4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.344616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.370303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.379827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.381609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.382392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.382711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.383014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.385250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.394630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a434de79e074483291f8689f729a22f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.398567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.400395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.403961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a434de79e074483291f8689f729a22f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.404694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a434de79e074483291f8689f729a22f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.411779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 955/1898] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:32.412480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.413820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.416698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a434de79e074483291f8689f729a22f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.417376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a434de79e074483291f8689f729a22f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:32.432768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 944/1899] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:32 2022] DELETE /identity/v3/users/a434de79e074483291f8689f729a22f4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:32.433838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.437564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed550-ba11-4411-8388-96209102a30c service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.439606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.511074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.511989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.512270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.512551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] PATH_INFO: `/v3/projects/14b49d73dfa341b381e39039b8e74767/users/f8af73ce7d2e48468bcba6b9ac39c821/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.528674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b49d73dfa341b381e39039b8e74767, user_id=f8af73ce7d2e48468bcba6b9ac39c821, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.531414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.535759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c14969-28b4-4b19-98f2-70387cc0b940 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.541233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.541891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.542176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.542549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.543231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.545038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.559948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 945/1901] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:32.559948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.560979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 956/1901] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:32 2022] PUT /identity/v3/projects/14b49d73dfa341b381e39039b8e74767/users/f8af73ce7d2e48468bcba6b9ac39c821/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:32.561959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.562064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.647661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d02b2-96e5-4386-bb90-08a61a5be051 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.651987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267f53c-0788-4d56-aefb-c277787b5e54 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.653118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.654079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.654414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.654609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] PATH_INFO: `/v3/projects/5df4e019ed5b4026a5cd1b431902793c/users/014de98d52e04dfba98dbbdc7f97fd0c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.672823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df4e019ed5b4026a5cd1b431902793c, user_id=014de98d52e04dfba98dbbdc7f97fd0c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.676738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5248233-8598-4b4e-bb44-7d2df4645494 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.716497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 957/1902] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:32 2022] PUT /identity/v3/projects/5df4e019ed5b4026a5cd1b431902793c/users/014de98d52e04dfba98dbbdc7f97fd0c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:32.716497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.716497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.716497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.738415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.763058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84291363651455985c8b65e5e3375d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe6b24733f7244afa86ae30de81a45df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.764370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.764874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.765362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.766350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:32.767534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.770387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7d23e-158b-46b4-bc8e-b188d2f37720 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:32.774183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] MFA Rules not processed for user `f8af73ce7d2e48468bcba6b9ac39c821`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:32.776218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.792050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 946/1903] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:32.793863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.794815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.795831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.830718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:32.848775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] MFA Rules not processed for user `014de98d52e04dfba98dbbdc7f97fd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:32.850971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.860220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 958/1904] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:32.861112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.947091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb95269-9b12-4852-8cc3-350c62752128 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:32.954451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 947/1905] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:32.954451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:32.954451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:32.957218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84291363651455985c8b65e5e3375d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe6b24733f7244afa86ae30de81a45df', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:32.958117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:32.958425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:32.958703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] PATH_INFO: `/v3/projects/0c86cdfc84044e23ad54e20233554fd4` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:32.959945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0c86cdfc84044e23ad54e20233554fd4)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:32.961868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.017935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c86cdfc84044e23ad54e20233554fd4', 'request_id': 'req-a8d77931-0a7a-408f-b6e2-3e8755264f12'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.029670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c86cdfc84044e23ad54e20233554fd4', 'request_id': 'req-a8d77931-0a7a-408f-b6e2-3e8755264f12'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.035405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a8477-1647-460a-b37c-2f1b7ed44543 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.043310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c86cdfc84044e23ad54e20233554fd4', 'request_id': 'req-a8d77931-0a7a-408f-b6e2-3e8755264f12'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.057638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] The token cache is being invalidate because project 0c86cdfc84044e23ad54e20233554fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:33.057946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.063046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.064942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 959/1906] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:32 2022] DELETE /identity/v3/projects/0c86cdfc84044e23ad54e20233554fd4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:33.065815 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.067352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.125944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.126861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.126861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.127149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.127854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.130123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.172957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 948/1907] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:32 2022] POST /identity/v3/projects => generated 365 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:33.174117 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.177209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.183905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d77931-0a7a-408f-b6e2-3e8755264f12 tempest-VolumeQuotaClassesTest-1578115484 tempest-VolumeQuotaClassesTest-1578115484-project-admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.187380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.226009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.272699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13450c3a-fa43-4b1e-9787-20cfc4fb3e29 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.277016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.277912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.278068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.278483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.278615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.278895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.279002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] PATH_INFO: `/v3/projects/7bf8ac2883264666805847f650fe2c90` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.279169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.279872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.280450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf8ac2883264666805847f650fe2c90)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.282484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.282600 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.320236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 949/1908] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:33.321522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.327834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.347644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf8ac2883264666805847f650fe2c90', 'request_id': 'req-a6f44189-e555-4e7d-aa2e-e16c1490bce3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.356421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf8ac2883264666805847f650fe2c90', 'request_id': 'req-a6f44189-e555-4e7d-aa2e-e16c1490bce3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.363048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf8ac2883264666805847f650fe2c90', 'request_id': 'req-a6f44189-e555-4e7d-aa2e-e16c1490bce3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.373205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] The token cache is being invalidate because project 7bf8ac2883264666805847f650fe2c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:33.373488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.374131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.375733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 960/1909] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:33 2022] DELETE /identity/v3/projects/7bf8ac2883264666805847f650fe2c90 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:33.376528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.377727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.404453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88f60-eee3-4dfb-8a2c-6e265bd66329 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.441086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f44189-e555-4e7d-aa2e-e16c1490bce3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.446696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.447379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.447655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.447927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.448623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.450578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.465914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.466864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.467212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.467567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.468329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.470420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a220ad5-47b3-42dd-812f-c65864461571 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.472764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 961/1910] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:33.473463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.479122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.493881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 950/1911] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:33.494387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4ec5e-206a-4ab7-9f85-d9683203614d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.495283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.495580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.496003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.500106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.501387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.502075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.503168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] PATH_INFO: `/v3/users/8a9498858dfa4015933f9545afddd148` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.504586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9498858dfa4015933f9545afddd148)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.506668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.515400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:33.516653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.530574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 951/1912] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:33.531703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.545610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04756cc0-adc7-4a1c-99ad-564072e1372e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.550990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.551854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.552365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.552851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.552992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.554118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.556374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.564913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9498858dfa4015933f9545afddd148'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.570181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 952/1913] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:33.571833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.574517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.575150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9498858dfa4015933f9545afddd148'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.575898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9498858dfa4015933f9545afddd148'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.590943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa3966-e256-4b26-879b-d0b1985277e9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.592042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9498858dfa4015933f9545afddd148'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.593393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4ebd69ff-7da8-4354-a700-5f295e141c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9498858dfa4015933f9545afddd148'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:33.596489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.597240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.597555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.597787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.598796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.600566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.606671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.611763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 962/1914] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:33 2022] DELETE /identity/v3/users/8a9498858dfa4015933f9545afddd148 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:33.613255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cdde-7474-42ae-a882-86a630153d33 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.613562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cdde-7474-42ae-a882-86a630153d33 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.613822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039cdde-7474-42ae-a882-86a630153d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.616880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 953/1915] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:33.617847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.620687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.634141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f50f62-1fa8-43d0-8227-0b90469951e6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.637840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7039cdde-7474-42ae-a882-86a630153d33 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:33.639115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7039cdde-7474-42ae-a882-86a630153d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.695799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.696692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.696970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.697289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] PATH_INFO: `/v3/projects/093d6c6d34224f949c32517089856a6d/users/a26881d95b0d497d9ffaebe87cb315ae/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.700419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 963/1916] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:33.701779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.702129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.702502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.712857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093d6c6d34224f949c32517089856a6d, user_id=a26881d95b0d497d9ffaebe87cb315ae, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.715966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.721111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:33.722381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.733503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 964/1917] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:33.734388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.734954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 954/1918] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:33 2022] PUT /identity/v3/projects/093d6c6d34224f949c32517089856a6d/users/a26881d95b0d497d9ffaebe87cb315ae/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:33.735870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.738502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.750284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac11df-54d1-4aa5-986a-2ed91a4e49c2 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.811153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.812055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.812730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.812730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] PATH_INFO: `/v3/projects/ef1206b8abe04ef1af475636ba08b15f/users/a8a5b88eb21247b78e78ea6fb113b6d0/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.816599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960abe46-e8fe-4270-80bb-ac0ad4316153 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.821733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.822436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.822738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.822991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.823718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:33.824019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.825629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.829153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1206b8abe04ef1af475636ba08b15f, user_id=a8a5b88eb21247b78e78ea6fb113b6d0, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.831944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.839426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 955/1919] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:33.844202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.848204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 965/1920] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:33 2022] PUT /identity/v3/projects/ef1206b8abe04ef1af475636ba08b15f/users/a8a5b88eb21247b78e78ea6fb113b6d0/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:33.848785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.850378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:33.854822 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2fd61-e30c-4678-9be4-f8add96780e3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.890705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.919213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.919576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c298c-18e5-4c53-8f58-cf9d54aab316 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.919886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.920197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.920532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.921238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.922835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.923156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.923794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.924103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.924340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.933418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 956/1921] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:33.934359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.937545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.939025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.944680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93edfc1-6dce-4af9-b8da-1810dc5f319a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.948847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 966/1922] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:33.949389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.949630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.950038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.950412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.950692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.951660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.953825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.958868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c437-ee13-4b29-9d1c-f4e4bb3774ec service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.963460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:33.964152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:33.964390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:33.965383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:33.965741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:33.966921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:33.967357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:33.979823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 967/1923] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:33.980697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:33.991088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6e1c-82ea-4e15-8514-ad1b5498c3a3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:33.992419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 957/1924] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:33 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:33.993364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.068913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff500656-d034-40e7-b6ca-b72fad5c0b98 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.075525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.075882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.076422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.077194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.077290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.077376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] PATH_INFO: `/v3/projects/093d6c6d34224f949c32517089856a6d/users/a26881d95b0d497d9ffaebe87cb315ae/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.077473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.077557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] PATH_INFO: `/v3/projects/ef1206b8abe04ef1af475636ba08b15f/users/a8a5b88eb21247b78e78ea6fb113b6d0/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.079076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=093d6c6d34224f949c32517089856a6d, user_id=a26881d95b0d497d9ffaebe87cb315ae, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.080021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1206b8abe04ef1af475636ba08b15f, user_id=a8a5b88eb21247b78e78ea6fb113b6d0, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.081845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.084817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20443097-efbb-41fd-ad82-94151bc9df8e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.095312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 968/1925] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:33 2022] PUT /identity/v3/projects/093d6c6d34224f949c32517089856a6d/users/a26881d95b0d497d9ffaebe87cb315ae/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:34.098982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 958/1926] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:33 2022] PUT /identity/v3/projects/ef1206b8abe04ef1af475636ba08b15f/users/a8a5b88eb21247b78e78ea6fb113b6d0/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:34.105228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.107934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333afa4-beff-46ad-ac59-7313b59a988c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.108678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333afa4-beff-46ad-ac59-7313b59a988c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.109021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333afa4-beff-46ad-ac59-7313b59a988c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.142734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8333afa4-beff-46ad-ac59-7313b59a988c None None] MFA Rules not processed for user `a26881d95b0d497d9ffaebe87cb315ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:34.144285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8333afa4-beff-46ad-ac59-7313b59a988c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.187905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b0b19-3572-492c-aefb-fc83006144cd admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.234274 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 959/1927] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:34.236041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.236463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.236910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.280664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] MFA Rules not processed for user `a8a5b88eb21247b78e78ea6fb113b6d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:34.285331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.319791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.320635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.321015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.321395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.322228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.324465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.334989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:34.357160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:34.364599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 969/1928] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:34 2022] POST /identity/v3/users => generated 289 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:34.364599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.367831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.369220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 960/1929] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:34.370234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.392291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140a2dc-8f72-4873-bb4e-c09a23e5368a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.398413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.399252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.399600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.400097 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.401053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.402875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.424801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 961/1930] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:34.424801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.424801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.460457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1ae5-9057-4049-bb47-dfdf2d500941 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.466365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.467261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.468039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.468472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.483808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.485361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.522544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 970/1931] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:34.524284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.531984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46415cec-29ce-4347-9dac-418a5de61af6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.545390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.546507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.547027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.547559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.547709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990eab6-671d-4022-a4eb-aae2f45095d9 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.557067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.557747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.558124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.558284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.559101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.560696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.567891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.569704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.572332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 971/1932] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:34.573189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.576423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.584830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 962/1933] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:34.585843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.591438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5342b-aae3-4550-80b5-ea97d0be56e1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.600785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.601262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54f1f9-7663-4503-9f0f-f06a8b41a6cb service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.602031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.602405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.602683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] PATH_INFO: `/v3/projects/d9f94c50ecf941e2999f231bce522081` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.611608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f94c50ecf941e2999f231bce522081)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.613726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.617669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.618718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.619133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.619596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] PATH_INFO: `/v3/projects/695df26fb22c4241af9fd330ecd3db9e/users/648faa02a82b4186b284e926087be5c8/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.637136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] RBAC: Authorizing `identity:create_grant(project_id=695df26fb22c4241af9fd330ecd3db9e, user_id=648faa02a82b4186b284e926087be5c8, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.640207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d0c1a-d382-446f-9a83-35ca8591688f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.670698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 963/1934] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:34 2022] PUT /identity/v3/projects/695df26fb22c4241af9fd330ecd3db9e/users/648faa02a82b4186b284e926087be5c8/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:34.684448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f94c50ecf941e2999f231bce522081', 'request_id': 'req-b6342095-0adb-4a4e-aa02-9dcc58292ef3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:34.684942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.685304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.685585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.699096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f94c50ecf941e2999f231bce522081', 'request_id': 'req-b6342095-0adb-4a4e-aa02-9dcc58292ef3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:34.706970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f94c50ecf941e2999f231bce522081', 'request_id': 'req-b6342095-0adb-4a4e-aa02-9dcc58292ef3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:34.715602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] MFA Rules not processed for user `648faa02a82b4186b284e926087be5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:34.716833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.720269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] The token cache is being invalidate because project d9f94c50ecf941e2999f231bce522081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:34.720557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:34.721423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:34.726491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 972/1935] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:34 2022] DELETE /identity/v3/projects/d9f94c50ecf941e2999f231bce522081 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:34.726491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.726491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.812596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6342095-0adb-4a4e-aa02-9dcc58292ef3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.819618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 964/1936] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:34.819618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.819899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.820007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.820093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.820093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.820471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.825222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:34.861516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:34.926763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:34.944819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.946676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.968690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228a31b-b005-46a6-92d7-cb02147096f2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:34.977145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:34.978143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:34.978365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 973/1937] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:34.978572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:34.978917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] PATH_INFO: `/v3/users/f84291363651455985c8b65e5e3375d9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:34.980298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84291363651455985c8b65e5e3375d9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:34.980714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:34.984615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:34.985543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.017445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f4aef-82ab-43de-b455-de78c1b8abbc service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.033575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.034529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.034913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.035293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.036313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.038443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.088730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 974/1938] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:34 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:35.090608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.100442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84291363651455985c8b65e5e3375d9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.122758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84291363651455985c8b65e5e3375d9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.123614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84291363651455985c8b65e5e3375d9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.149784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84291363651455985c8b65e5e3375d9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.150906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84291363651455985c8b65e5e3375d9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.187130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 965/1939] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:34 2022] DELETE /identity/v3/users/f84291363651455985c8b65e5e3375d9 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:35.245431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ece210-d7fc-484c-8b5e-d7f13f19cfe2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.398912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.400014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.400813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.400813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.401990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.404551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.410506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.412931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.440103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ff328-d8a2-494c-9812-2b4f8a2096f9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.450085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.450915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.451196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.451611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] PATH_INFO: `/v3/users/014de98d52e04dfba98dbbdc7f97fd0c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.456250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] RBAC: Authorizing `identity:delete_user(user_id=014de98d52e04dfba98dbbdc7f97fd0c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.456647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 975/1940] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:35 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:35.457749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.462530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.465107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.497427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f339d-8282-465b-8c03-632285c9ac0a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.506106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.506835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.507147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.507440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.508181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.523089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.539404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014de98d52e04dfba98dbbdc7f97fd0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.542809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 976/1941] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:35.546517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.560504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d059e4-cc13-4e1f-a4f8-d6084b2de771 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.567039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014de98d52e04dfba98dbbdc7f97fd0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.568265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014de98d52e04dfba98dbbdc7f97fd0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.571184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.573382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.573382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.573681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.579578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.583768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.590242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014de98d52e04dfba98dbbdc7f97fd0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.590702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f564f444-dc1b-4328-9b76-3add51d3c76f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014de98d52e04dfba98dbbdc7f97fd0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:35.611612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 977/1942] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:35.613622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.626533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.632654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 966/1943] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:35 2022] DELETE /identity/v3/users/014de98d52e04dfba98dbbdc7f97fd0c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:35.633733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b512-36a0-4ef9-8552-829d1bc8872f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.634110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b512-36a0-4ef9-8552-829d1bc8872f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.634276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676b512-36a0-4ef9-8552-829d1bc8872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.665032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c341a0a-f71d-4e81-9ab9-98f78fefc5d8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.682789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:35.683791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0676b512-36a0-4ef9-8552-829d1bc8872f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:35.687138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0676b512-36a0-4ef9-8552-829d1bc8872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.773418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.776139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.776506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.777457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] PATH_INFO: `/v3/projects/10773d86a70b41eba14c3faaf10b4871/users/7d84222dffc44024923534abd40e0a21/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.791998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:35.803024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] RBAC: Authorizing `identity:create_grant(project_id=10773d86a70b41eba14c3faaf10b4871, user_id=7d84222dffc44024923534abd40e0a21, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:35.806528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:35.828238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 967/1944] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:35.829169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 978/1945] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:35 2022] PUT /identity/v3/projects/10773d86a70b41eba14c3faaf10b4871/users/7d84222dffc44024923534abd40e0a21/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:35.843938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.844538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.845236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:35.858722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.860758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.881269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] MFA Rules not processed for user `7d84222dffc44024923534abd40e0a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:35.883164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:35.968254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 968/1946] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:35.969824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:35.983035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356bca87-b49f-4ed9-a8cf-2e7eaf8e540b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:35.987543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:35.988524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:35.988948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:35.989348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.092312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.093205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03210b87-6db2-4302-a018-d6f95ec5feda None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.093839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.100632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.101556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.101952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.102479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.103295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.105526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.107059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 979/1947] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:36.108676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.110789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.125805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87d009-d642-4b66-8f2a-cda50e9e3150 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.132607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.133475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.133758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.134024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] PATH_INFO: `/v3/projects/fe6b24733f7244afa86ae30de81a45df` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.135254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6b24733f7244afa86ae30de81a45df)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.137190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.138993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 969/1948] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:35 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:36.142839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.196393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:36.205671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6b24733f7244afa86ae30de81a45df', 'request_id': 'req-5a22f184-4f6c-4db2-8219-cbcbe4479880'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.218817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6b24733f7244afa86ae30de81a45df', 'request_id': 'req-5a22f184-4f6c-4db2-8219-cbcbe4479880'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.231237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6b24733f7244afa86ae30de81a45df', 'request_id': 'req-5a22f184-4f6c-4db2-8219-cbcbe4479880'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.231943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490c87-af0f-4b01-ae10-bfecf8c6d23a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.237461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:36.245513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] The token cache is being invalidate because project fe6b24733f7244afa86ae30de81a45df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:36.246004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.247884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5a22f184-4f6c-4db2-8219-cbcbe4479880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.254287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 980/1949] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:36 2022] DELETE /identity/v3/projects/fe6b24733f7244afa86ae30de81a45df => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.260788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb3708-8cf1-43b7-bee2-dc35ec8d3483 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.260788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb3708-8cf1-43b7-bee2-dc35ec8d3483 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.260788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb3708-8cf1-43b7-bee2-dc35ec8d3483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.280662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3beb3708-8cf1-43b7-bee2-dc35ec8d3483 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:36.282107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3beb3708-8cf1-43b7-bee2-dc35ec8d3483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.327443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.328333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.328666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.329021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.329794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.333990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.373733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 981/1950] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:36.382417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.382847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.383290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.392587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 970/1951] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/users => generated 295 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:36.399348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.401982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.409792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:36.411251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.420932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 982/1952] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:36.426842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.438709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e0126-0fd6-4366-8019-6dd94bb02363 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.446452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f834af-ee0d-4a4c-a9fb-b6f0e82a0f85 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.459328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.459328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.459328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.459328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.460209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.460209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.461440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.461766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.462066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.462377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.463175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:36.463488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.465496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.469827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 971/1953] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:36.475385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.482368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 983/1954] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:36.484023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.485604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dd1a6-737f-4cfe-83ab-8cfd1ca66d25 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.490622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.491516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.491801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.492084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] PATH_INFO: `/v3/projects/75a1596d9a5a4094bbd72004de9789a1/users/7483b38fdcf4493986c2efd6a91be278/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.494728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fa661-e81e-4c8c-8f28-353eaf306a71 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.499364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.500331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.500685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.501025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.501789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.504160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.509045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a1596d9a5a4094bbd72004de9789a1, user_id=7483b38fdcf4493986c2efd6a91be278, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.511745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.533936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 972/1955] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:36 2022] PUT /identity/v3/projects/75a1596d9a5a4094bbd72004de9789a1/users/7483b38fdcf4493986c2efd6a91be278/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.537790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 984/1956] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:36.538310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.540851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.607891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a961800-1e68-4bde-8076-d3314a6b8899 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.612621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.613318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.613583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.613966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.614605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.616666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.620608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7d526-82f1-4ff6-a640-e18df09ba798 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.627180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.628343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.628343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.628745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.629569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.631560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.641816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 985/1957] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:36.641816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.641816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.645831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 973/1958] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:36.646622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.654701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b7716-1845-4045-b60e-5b3f5857773d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.656602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd48ea-bc20-423b-8dc1-de16edf8b864 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.659848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.660876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.661228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.661571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] PATH_INFO: `/v3/projects/5df4e019ed5b4026a5cd1b431902793c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.661679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.662277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.662606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.662878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.663006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df4e019ed5b4026a5cd1b431902793c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.663599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.665055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.665200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.681942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:36.689043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 974/1959] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:36.689043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.699269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba34d67-6208-4804-a4a6-952f67b99af9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.704760 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.706022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.706231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.707244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] PATH_INFO: `/v3/projects/75a1596d9a5a4094bbd72004de9789a1/users/7483b38fdcf4493986c2efd6a91be278/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.709241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a1596d9a5a4094bbd72004de9789a1, user_id=7483b38fdcf4493986c2efd6a91be278, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.711942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.724805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df4e019ed5b4026a5cd1b431902793c', 'request_id': 'req-4dc7e8bc-36c2-407c-a63c-500922f8386e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.725483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 975/1960] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:36 2022] PUT /identity/v3/projects/75a1596d9a5a4094bbd72004de9789a1/users/7483b38fdcf4493986c2efd6a91be278/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.726402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.743080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df4e019ed5b4026a5cd1b431902793c', 'request_id': 'req-4dc7e8bc-36c2-407c-a63c-500922f8386e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.750946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df4e019ed5b4026a5cd1b431902793c', 'request_id': 'req-4dc7e8bc-36c2-407c-a63c-500922f8386e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.762333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] The token cache is being invalidate because project 5df4e019ed5b4026a5cd1b431902793c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:36.763125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.763318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-4dc7e8bc-36c2-407c-a63c-500922f8386e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.767954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 986/1961] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:36 2022] DELETE /identity/v3/projects/5df4e019ed5b4026a5cd1b431902793c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.768087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:36.768185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.768185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.768185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.796216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] MFA Rules not processed for user `7483b38fdcf4493986c2efd6a91be278`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:36.797529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.801682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6f4b6-55d6-4a14-b61a-7c333d729df6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.806531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.807270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.807620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.807960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.808761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.810467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.821101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 976/1962] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:36.824898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.826725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.840870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c787464-c8d4-4a7e-8ef0-e16b2122726a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.845651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.846554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.846834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.847167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] PATH_INFO: `/v3/users/a26881d95b0d497d9ffaebe87cb315ae` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.848366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26881d95b0d497d9ffaebe87cb315ae)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.850287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.879402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 987/1963] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:36.880291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.881726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.902064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e36276-1d0a-4314-81bf-c3284be8f1d4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:36.907611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:36.908693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.909059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.909470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] PATH_INFO: `/v3/projects/78ba0efcdbc24870b2ad9598dd3c0a89/users/c0ea10fde2f14ad39da4141cdfa9fe26/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:36.925614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26881d95b0d497d9ffaebe87cb315ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.936008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26881d95b0d497d9ffaebe87cb315ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.937733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26881d95b0d497d9ffaebe87cb315ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.939434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ba0efcdbc24870b2ad9598dd3c0a89, user_id=c0ea10fde2f14ad39da4141cdfa9fe26, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:36.948893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:36.962501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26881d95b0d497d9ffaebe87cb315ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.963419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2f5fd2ca-5c86-4d9f-84b2-16d7e39b7a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26881d95b0d497d9ffaebe87cb315ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:36.969563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 988/1964] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:36 2022] PUT /identity/v3/projects/78ba0efcdbc24870b2ad9598dd3c0a89/users/c0ea10fde2f14ad39da4141cdfa9fe26/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.969563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:36.985756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 977/1965] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:36 2022] DELETE /identity/v3/users/a26881d95b0d497d9ffaebe87cb315ae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:36.985934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:36.986061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:36.986061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:36.986061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.036915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] MFA Rules not processed for user `c0ea10fde2f14ad39da4141cdfa9fe26`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:37.038298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.111244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2c15-6c3d-4c90-8cdb-9cf90c4e5bd7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.136984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:37.170122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 978/1966] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:37.174801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.177177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.191613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.192541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.192755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.193110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.195546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.226626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:37.278435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.280042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.283102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fbd08-0345-4649-b5ed-2302a9e3b64f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.291580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.292318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.292746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.293083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.293983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.296041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.296840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 989/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:37.350340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 979/1968] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/projects => generated 409 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:37.354128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.368418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7799607-e318-4960-b0f7-aa815f56a8b6 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.432883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.434271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.434665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.435122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.436090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.439224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.468170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.470040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.484957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 990/1969] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/users => generated 311 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:37.489086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.490497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe293f-7c00-4415-8b80-682b2eebd498 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.497405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.499300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.499300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.499492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] PATH_INFO: `/v3/projects/093d6c6d34224f949c32517089856a6d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.500846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093d6c6d34224f949c32517089856a6d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.504838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.515023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069bf37-265e-4c1b-a200-f6a92cf56bf1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.529354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.530874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.531155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.531434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.532269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.537932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.557548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 991/1970] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:37.563653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.582814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cf058-e258-46cb-b777-b9f5e2170a8f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.584473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093d6c6d34224f949c32517089856a6d', 'request_id': 'req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:37.597173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093d6c6d34224f949c32517089856a6d', 'request_id': 'req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:37.606219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093d6c6d34224f949c32517089856a6d', 'request_id': 'req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:37.621539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] The token cache is being invalidate because project 093d6c6d34224f949c32517089856a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:37.621829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:37.622664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1b3637d7-12d7-4ddd-8d7d-c99bc3fd46a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:37.624737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 980/1971] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:37 2022] DELETE /identity/v3/projects/093d6c6d34224f949c32517089856a6d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:37.630561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82447-5f53-488d-8fa6-8a1849a618b2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.630929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82447-5f53-488d-8fa6-8a1849a618b2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.631270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82447-5f53-488d-8fa6-8a1849a618b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.652602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6ac82447-5f53-488d-8fa6-8a1849a618b2 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:37.659794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6ac82447-5f53-488d-8fa6-8a1849a618b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.700081 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:37.724282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.725714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.727545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.729698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] PATH_INFO: `/v3/projects/6df08522cb4442c499616ff63ad85ea6/users/a0476a2f0fbb4e9abffd95414e35d02d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.773646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df08522cb4442c499616ff63ad85ea6, user_id=a0476a2f0fbb4e9abffd95414e35d02d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.778483 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.808837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 992/1972] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:37 2022] PUT /identity/v3/projects/6df08522cb4442c499616ff63ad85ea6/users/a0476a2f0fbb4e9abffd95414e35d02d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:37.813128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.817087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.823055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 981/1973] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:37.833188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.833621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.834064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.856495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:37.857801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:37.898367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:37.905838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1245e8c-561c-4dea-b64a-c88064fb4fa6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.912197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.913043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.913322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.913597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.914393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.916085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.930439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 993/1974] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:37.934145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 982/1975] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:37.935142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.938461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.946864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569bd53-b308-43bc-a454-5dea79ff1973 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.948444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99427ead-08b3-4bc6-a425-abb8ffb73f8c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:37.952158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.953053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.953328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.953626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] PATH_INFO: `/v3/projects/6df08522cb4442c499616ff63ad85ea6/users/a0476a2f0fbb4e9abffd95414e35d02d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.953905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:37.954622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.954903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.955204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:37.955758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df08522cb4442c499616ff63ad85ea6, user_id=a0476a2f0fbb4e9abffd95414e35d02d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.955945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:37.956255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:37.957985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2219-0c08-4108-8fe4-40dd592309a0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.958718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:37.969610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 994/1976] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:37 2022] PUT /identity/v3/projects/6df08522cb4442c499616ff63ad85ea6/users/a0476a2f0fbb4e9abffd95414e35d02d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:37.971103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 983/1977] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:37.975529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:37.981406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:37.981696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:37.982007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.008732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] MFA Rules not processed for user `a0476a2f0fbb4e9abffd95414e35d02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:38.009866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:38.040935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530baec-a89e-404c-b3d9-ab2d8773cc09 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.046536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.047359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.047664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.047886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.048818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.051543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.088267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 984/1978] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:38.088401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 995/1979] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:37 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:38.093320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.100213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.101923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:38.109729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9eb7f-df10-46aa-9825-b4fa4d78c0c9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.158824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:38.176578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfafb0a-6038-4a39-9f7f-04badf0a9f9e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.183558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.183724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.184290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.184396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.184706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.184821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.185107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.185208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.185921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.187972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.218360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 985/1980] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:38 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:38.219492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.231932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252bdaaf-9241-467c-bd2a-e8e90ec91d2d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.237876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.238758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.239210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.239842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.240732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.243268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.258839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 986/1981] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:38.260008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.271143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173bd7d-6820-4a60-821c-2831f97ae29d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.277012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.277929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.278348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.278719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.279967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.281560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.293709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 987/1982] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:38.298893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.300684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f385fe-575b-432e-8efd-6393b3e7c86f service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.301220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.312078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 996/1983] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:38 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:38.331692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f1daa-0b07-488d-8ea2-91255cc08d95 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.338077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.339468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.340355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.340804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] PATH_INFO: `/v3/projects/6a8d6b5c315f4d6c891257380341ab4e/users/5f201381ff514b3c9c8862673426de1b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.373998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8d6b5c315f4d6c891257380341ab4e, user_id=5f201381ff514b3c9c8862673426de1b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.377369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.382820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:38.407461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 988/1984] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:38 2022] PUT /identity/v3/projects/6a8d6b5c315f4d6c891257380341ab4e/users/5f201381ff514b3c9c8862673426de1b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:38.418853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.419248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.419623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.479194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] MFA Rules not processed for user `5f201381ff514b3c9c8862673426de1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:38.480452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:38.616434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.628896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 997/1985] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:38.633834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.688893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34a11b-c5eb-4cce-9458-950a2f71a4a5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.694930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.695687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.696032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.696402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.699100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:38.775324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed79d-98c7-43f1-a8bd-d147000f2f75 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.782698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.783252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.783560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.784033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.785076 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.787873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.797625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.799691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.811712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 989/1986] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:38 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:38.833956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 998/1987] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:38 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:38.834421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.846657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b837aa-cb3b-4d25-9510-0dca0e8af816 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.916149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.917863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.918670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.919592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.920823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.924670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.956076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 990/1988] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:38 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:38.960773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:38.973589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2aed6-3e44-49a1-9ec8-9fe70539a49e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:38.979141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:38.979835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:38.980120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:38.980381 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:38.981126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:38.982966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:38.984488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:38.997056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 999/1989] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:39.002423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.013210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b468a9-8df7-4eb1-b9bc-ab3d41c45581 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.018632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.019517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.019817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.020073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] PATH_INFO: `/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a/users/42d4d6d9caf94d4f93652d9a7cd5df4a/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.041317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61f67c0a4f94ac48b8ef3e60ecd383a, user_id=42d4d6d9caf94d4f93652d9a7cd5df4a, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.044090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.062164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 991/1990] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:39 2022] PUT /identity/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a/users/42d4d6d9caf94d4f93652d9a7cd5df4a/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:39.066385 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.077009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04be02-6499-4373-9982-eae54da98aee admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.147644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.149051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.149051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.149274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.150140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.151967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403fb00c-7344-44a3-bfb7-0d7e21a717eb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.174879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1000/1991] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:39.179248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.189864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac21db0-9651-4f60-9eb6-a749f13dc33d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.196242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.197617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.197899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.198222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] PATH_INFO: `/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a/users/42d4d6d9caf94d4f93652d9a7cd5df4a/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.200337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61f67c0a4f94ac48b8ef3e60ecd383a, user_id=42d4d6d9caf94d4f93652d9a7cd5df4a, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.203018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.214006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 992/1992] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:39 2022] PUT /identity/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a/users/42d4d6d9caf94d4f93652d9a7cd5df4a/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:39.225405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.225674 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.225952 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.257357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.258856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:39.260884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] MFA Rules not processed for user `42d4d6d9caf94d4f93652d9a7cd5df4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:39.262242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:39.264648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:39.374725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1001/1993] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:39.382196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbf066-7aaa-495b-bf21-890f0f85c29d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.387145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.388438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.388821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.389104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.392272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:39.412576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.423083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101136ed-8f7a-4ae5-97d4-f6b4e20efbbf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.426761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.427413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.427692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.427963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.441557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.443507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.454083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1002/1994] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:39.493146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.498276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.528874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 993/1995] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:39.686551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:39.698532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c285fc-0a9d-4875-afee-72d5c13cbc8c service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:39.704966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:39.705635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:39.705979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:39.706430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:39.799538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:39.801083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:39.811511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1003/1996] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:40.606245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:40.685800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:40.690954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:40.705259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08e6cd-da55-49da-8512-94f41f121a68 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:40.713457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:40.715736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:40.716025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:40.716311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:40.723543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:40.727448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ece41eb-3c44-4581-82c9-8e4d6c679991 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:40.735953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:40.738567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:40.739165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:40.739661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] PATH_INFO: `/v3/users/5f201381ff514b3c9c8862673426de1b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:40.741130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f201381ff514b3c9c8862673426de1b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:40.743952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:40.815214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f201381ff514b3c9c8862673426de1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:40.825774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:40.827795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e2b6-8e25-4779-b3a7-0a1e82436edc service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:40.842755 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1004/1997] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:40.851609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f201381ff514b3c9c8862673426de1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:40.854362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f201381ff514b3c9c8862673426de1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:40.867871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:40.881588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f201381ff514b3c9c8862673426de1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:40.882677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f201381ff514b3c9c8862673426de1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:40.911747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 994/1998] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:40 2022] DELETE /identity/v3/users/5f201381ff514b3c9c8862673426de1b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:40.917316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:40.917316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:40.917316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:40.940054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:40.941407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.038364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1005/1999] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:41.048487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:41.050227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.118523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506b393-7ec5-494b-9e95-5cdaa87fc64b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:41.122752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:41.123545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:41.123917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:41.124279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:41.136466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:41.137920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678353e6-bed2-4909-ab10-baff6bee92bc service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:41.148085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:41.148453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 995/2000] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:41.149759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.162003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5754-7f65-4cab-bd4c-6c699e699b90 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:41.166761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:41.167841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:41.168124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:41.168418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] PATH_INFO: `/v3/users/f8af73ce7d2e48468bcba6b9ac39c821` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:41.169717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8af73ce7d2e48468bcba6b9ac39c821)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:41.171985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:41.231654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8af73ce7d2e48468bcba6b9ac39c821'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:41.240989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8af73ce7d2e48468bcba6b9ac39c821'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:41.241726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8af73ce7d2e48468bcba6b9ac39c821'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:41.258137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8af73ce7d2e48468bcba6b9ac39c821'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:41.259095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8af73ce7d2e48468bcba6b9ac39c821'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:41.273738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1006/2001] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:41 2022] DELETE /identity/v3/users/f8af73ce7d2e48468bcba6b9ac39c821 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:41.280914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:41.281625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:41.282653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:41.306145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:41.307652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.387770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 996/2002] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:41.396990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:41.398880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.431049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:41.500231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50dc7c3-b7a5-4d12-8095-0a002ca21a0f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:41.505101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:41.505766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:41.506040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:41.506727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:41.524876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:41.526365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:41.559280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1007/2003] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:41.922674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:41.924120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:41.936751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edba0df-fc60-40dc-98b3-8c6e23aa8176 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:41.942536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:41.943424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:41.943708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:41.943982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] PATH_INFO: `/v3/projects/6a8d6b5c315f4d6c891257380341ab4e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:41.945189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8d6b5c315f4d6c891257380341ab4e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:41.947193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:42.003902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8d6b5c315f4d6c891257380341ab4e', 'request_id': 'req-a918327f-7e1d-426d-b270-09549757fd67'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.015636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8d6b5c315f4d6c891257380341ab4e', 'request_id': 'req-a918327f-7e1d-426d-b270-09549757fd67'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.025780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8d6b5c315f4d6c891257380341ab4e', 'request_id': 'req-a918327f-7e1d-426d-b270-09549757fd67'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.047232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] The token cache is being invalidate because project 6a8d6b5c315f4d6c891257380341ab4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:42.047488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.048598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.051232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 997/2004] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:41 2022] DELETE /identity/v3/projects/6a8d6b5c315f4d6c891257380341ab4e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:42.056466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:42.069421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883c3aa-f5a4-41b8-9e7c-112edb23f568 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:42.136179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:42.137238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.137609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.138120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] PATH_INFO: `/v3/users/42d4d6d9caf94d4f93652d9a7cd5df4a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.139452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=42d4d6d9caf94d4f93652d9a7cd5df4a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:42.141530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:42.203893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42d4d6d9caf94d4f93652d9a7cd5df4a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.216723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42d4d6d9caf94d4f93652d9a7cd5df4a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.217578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42d4d6d9caf94d4f93652d9a7cd5df4a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.231491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42d4d6d9caf94d4f93652d9a7cd5df4a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.232279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42d4d6d9caf94d4f93652d9a7cd5df4a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.248360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1008/2005] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:42 2022] DELETE /identity/v3/users/42d4d6d9caf94d4f93652d9a7cd5df4a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:42.287440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:42.288950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:42.302149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:42.340233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918327f-7e1d-426d-b270-09549757fd67 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:42.415858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:42.417638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.417973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.418256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] PATH_INFO: `/v3/projects/14b49d73dfa341b381e39039b8e74767` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.420067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b49d73dfa341b381e39039b8e74767)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:42.422910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:42.485122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b49d73dfa341b381e39039b8e74767', 'request_id': 'req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.497567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b49d73dfa341b381e39039b8e74767', 'request_id': 'req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.506878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b49d73dfa341b381e39039b8e74767', 'request_id': 'req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.528184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] The token cache is being invalidate because project 14b49d73dfa341b381e39039b8e74767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:42.529001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.529470 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.534425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 998/2006] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:42 2022] DELETE /identity/v3/projects/14b49d73dfa341b381e39039b8e74767 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:42.535705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:42.537324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:42.552234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f3ea9-8cc3-4704-a135-2119ca0180ae admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:42.595839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:42.633407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:42.634520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.634874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.635198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] PATH_INFO: `/v3/users/a8a5b88eb21247b78e78ea6fb113b6d0` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.636484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a5b88eb21247b78e78ea6fb113b6d0)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:42.638698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:42.686053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:42.688071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:42.704487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4da67f-07c3-42d7-8a02-a9a5d2e8fd12 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:42.712006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:42.712898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a5b88eb21247b78e78ea6fb113b6d0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.713644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.714200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.715027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] PATH_INFO: `/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.716837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61f67c0a4f94ac48b8ef3e60ecd383a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:42.719716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:42.727876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a5b88eb21247b78e78ea6fb113b6d0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.728724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a5b88eb21247b78e78ea6fb113b6d0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.747265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a5b88eb21247b78e78ea6fb113b6d0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.748325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f254bb8e-9a79-4ca9-aa93-9e8837f20ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a5b88eb21247b78e78ea6fb113b6d0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.764456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1009/2007] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:42 2022] DELETE /identity/v3/users/a8a5b88eb21247b78e78ea6fb113b6d0 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:42.782305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61f67c0a4f94ac48b8ef3e60ecd383a', 'request_id': 'req-d1c4869c-1423-484c-abb0-e708f56f8e71'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.802700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61f67c0a4f94ac48b8ef3e60ecd383a', 'request_id': 'req-d1c4869c-1423-484c-abb0-e708f56f8e71'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.820177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61f67c0a4f94ac48b8ef3e60ecd383a', 'request_id': 'req-d1c4869c-1423-484c-abb0-e708f56f8e71'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.832712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] The token cache is being invalidate because project f61f67c0a4f94ac48b8ef3e60ecd383a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:42.832712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.833536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d1c4869c-1423-484c-abb0-e708f56f8e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:42.835212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 999/2008] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:42 2022] DELETE /identity/v3/projects/f61f67c0a4f94ac48b8ef3e60ecd383a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:42.841147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.841649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.841835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.863523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:42.864874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:42.964228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1010/2009] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:42.973520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:42.973982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:42.974074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:42.994477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:42.995712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.005376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1000/2010] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:43.009418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.020174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f8b7c-4758-41ef-88eb-1f086925a5f3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.025723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.026283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.026964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.027313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.028125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:43.028758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.030588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.043728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1011/2011] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:43.048385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.059848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8750a-dcce-4649-9cc2-666b9e84ba10 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.065130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.065922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.066289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.066656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.067654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.069697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.075124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:43.106512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1001/2012] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:43.110621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.119834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.123359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.130705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582640b2-985c-42f6-a87b-915161fbeff3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.179175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:43.206244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c3098-5d93-4796-a7f2-f9d3232e6363 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.209187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.209881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.210158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.210480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.211277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.211380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.212089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.212368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.212645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] PATH_INFO: `/v3/projects/ef1206b8abe04ef1af475636ba08b15f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.213152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.213988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1206b8abe04ef1af475636ba08b15f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.216318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.241759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1012/2013] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:43.242898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.260835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d24a6-d747-4d4e-be29-1b7a3afa93b2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.270560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1206b8abe04ef1af475636ba08b15f', 'request_id': 'req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:43.281740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1206b8abe04ef1af475636ba08b15f', 'request_id': 'req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:43.289158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1206b8abe04ef1af475636ba08b15f', 'request_id': 'req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:43.300182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] The token cache is being invalidate because project ef1206b8abe04ef1af475636ba08b15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:43.300469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:43.301315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3b77026-90a3-45b6-88e8-9d22d5ebc1ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:43.302942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1002/2014] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:43 2022] DELETE /identity/v3/projects/ef1206b8abe04ef1af475636ba08b15f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:43.308016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30bd1-7bf7-4dde-885e-dee8363e25a8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.308408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30bd1-7bf7-4dde-885e-dee8363e25a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.308744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba30bd1-7bf7-4dde-885e-dee8363e25a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.328936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0ba30bd1-7bf7-4dde-885e-dee8363e25a8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:43.330147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ba30bd1-7bf7-4dde-885e-dee8363e25a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.330547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.331495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.331885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.332427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.333273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.335024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.349156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1013/2015] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:43.350775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.352706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.358358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1003/2016] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:43.363246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.363651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.363994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.367761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ab98e-d798-4f7a-8ea2-1b0404d4930f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.373597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.374282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.374602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.375052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.375884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.377411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.384529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:43.385893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.391042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1014/2017] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:43.395303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.398022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1004/2018] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:43.401957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.410535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ed451-2ad7-4251-99b6-ea8dd745a647 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.415625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36762eb2-1032-420c-a57b-d76bae868e51 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.415814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.416723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.417006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.417281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] PATH_INFO: `/v3/projects/9f54a08332aa4a5d8ccb9e53c76158ea/users/82703d20c3834d148ce525b17a98367f/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.421177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.421957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.422344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.422847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.423690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:43.424089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.426908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.435492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f54a08332aa4a5d8ccb9e53c76158ea, user_id=82703d20c3834d148ce525b17a98367f, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.438407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1594b8d-cb1f-40bc-96f8-eb4d39808fcc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.448852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1005/2019] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:43.451289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.455945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1015/2020] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:43 2022] PUT /identity/v3/projects/9f54a08332aa4a5d8ccb9e53c76158ea/users/82703d20c3834d148ce525b17a98367f/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:43.467009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.467295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.467571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.469556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85f9a8-4f14-42e0-ae79-ad7e3772682f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.499065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] MFA Rules not processed for user `82703d20c3834d148ce525b17a98367f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:43.500402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:43.541866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:43.555967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.556976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.557324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.558078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.558485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.560480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.573393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1016/2021] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:43.577619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.590399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13533a9b-5dbb-4045-bd1e-2e25964154cb None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.599459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1006/2022] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:43.600414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.601681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.602915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.603194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.603469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.604175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.606116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.628331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:43.650340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1017/2023] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:43.654384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.678794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82269c-98c9-41e6-b088-da4000c92856 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.745188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50340c8-9820-4ea8-9a7e-0ce71f6d906e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.752089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.752819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.753466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.753750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.754526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.757082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.761196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.762195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.762542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.762925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.763767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.766025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.794087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1018/2024] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:43.800397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.804498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1007/2025] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:43 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:43.808902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.826658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea30ed-e6bf-4853-afcf-6784151c0668 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.827729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7dd98-0821-4eed-b265-af3e35f944ff admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.832701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.833562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.833847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.834194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.835609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.835780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.835965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.836444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.836809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.836916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.837713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.839457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.853396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1008/2026] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:43.855922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.856053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1019/2027] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:43.856896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.868915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b3866-97a4-4b37-964d-93f082df166f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.871050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401b27d-22fb-490a-a658-8b942b231532 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.876924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.877832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.878545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.878865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] PATH_INFO: `/v3/projects/b456a528c4b44d9ab152b91c72ff5143/users/2b2dac7de96f4b8b9f49c83f70354725/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.879184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.879956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.880330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.880699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.881974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.884816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.896111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b456a528c4b44d9ab152b91c72ff5143, user_id=2b2dac7de96f4b8b9f49c83f70354725, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.898554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:43.902658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1020/2028] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:43.903957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.916389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1009/2029] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:43 2022] PUT /identity/v3/projects/b456a528c4b44d9ab152b91c72ff5143/users/2b2dac7de96f4b8b9f49c83f70354725/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:43.917953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec057bf5-500c-40dd-9f77-26883b6cb90f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.920462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:43.966556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:43.983804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.984700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.984974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.985244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] PATH_INFO: `/v3/projects/b19521d94fcc490ebc947a9cbdd733ea/users/4b279e5cb18e484d8e4997d5d35b9935/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.990727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecaa54-a1a9-4240-accc-c5e61379920c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:43.995661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:43.996462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:43.996836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:43.997208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:43.998043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:43.999836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:44.002613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19521d94fcc490ebc947a9cbdd733ea, user_id=4b279e5cb18e484d8e4997d5d35b9935, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:44.005332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce9a77-2bf4-46a5-b381-d77494aa7c3f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:44.011174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1010/2030] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:44.015966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:44.020118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1021/2031] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:43 2022] PUT /identity/v3/projects/b19521d94fcc490ebc947a9cbdd733ea/users/4b279e5cb18e484d8e4997d5d35b9935/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:44.026174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f232da7-9dab-4b40-9283-39ae71a71568 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:44.031433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:44.031787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:44.032181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:44.062805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] MFA Rules not processed for user `4b279e5cb18e484d8e4997d5d35b9935`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:44.064033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:44.092406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:44.093610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:44.093991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:44.094413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] PATH_INFO: `/v3/projects/b456a528c4b44d9ab152b91c72ff5143/users/2b2dac7de96f4b8b9f49c83f70354725/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:44.096759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b456a528c4b44d9ab152b91c72ff5143, user_id=2b2dac7de96f4b8b9f49c83f70354725, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:44.100475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e70b26-62f7-4f0f-a34e-785bc3baf552 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:44.103362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:44.115673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1011/2032] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:44 2022] PUT /identity/v3/projects/b456a528c4b44d9ab152b91c72ff5143/users/2b2dac7de96f4b8b9f49c83f70354725/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:44.128615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:44.129007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:44.129399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:44.163237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1022/2033] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:44.170906 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] MFA Rules not processed for user `2b2dac7de96f4b8b9f49c83f70354725`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:44.172245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:44.176688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:44.179832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:44.251617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89233e7-7c45-402c-96a4-16225f3dfa54 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:44.269943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1012/2034] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:44.276805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:44.279109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:44.311221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:44.312570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:44.312570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:44.312570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:44.337285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f070a3-0874-4741-becf-febffc393dc1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:44.341453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:44.342173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:44.342545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:44.342881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:44.350314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:44.381176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:44.382691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:44.395846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1023/2035] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:44.406813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:44.408308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:44.422040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1013/2036] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:49.822331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:49.823846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:49.913264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:49.947875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:49.950595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:49.964092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7931b8c-6f4f-46cf-aad4-50a3f7934fb5 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:49.970493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:49.971205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:49.971489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:49.971762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:49.988569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bedc0-7f2c-4f5b-ae9a-184262d71404 service glance] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:49.993137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:49.994826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ff259-f305-4cb2-94ca-8cfe43080ff6 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:50.001482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:50.002423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:50.002708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:50.002983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] PATH_INFO: `/v3/users/c0ea10fde2f14ad39da4141cdfa9fe26` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:50.005758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ea10fde2f14ad39da4141cdfa9fe26)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:50.007387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:50.024377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1014/2037] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:50.107331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ea10fde2f14ad39da4141cdfa9fe26'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:50.135273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ea10fde2f14ad39da4141cdfa9fe26'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:50.136096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ea10fde2f14ad39da4141cdfa9fe26'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:50.175622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ea10fde2f14ad39da4141cdfa9fe26'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:50.178058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ea10fde2f14ad39da4141cdfa9fe26'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:50.206822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1024/2038] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:49 2022] DELETE /identity/v3/users/c0ea10fde2f14ad39da4141cdfa9fe26 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:50.210814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:50.211584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:50.212010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:50.242262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:50.243517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:50.355658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1015/2039] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:50.367797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:50.369983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:50.525096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6272d6-60d0-4357-a1e2-39827b531b41 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:50.528624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:50.529277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:50.529557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:50.529924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:50.557679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:50.559312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:50.596612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1025/2040] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:51.274949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:51.276878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:51.304759 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089250-08de-434a-ada1-6eea9c0e0b04 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:51.314855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:51.316443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:51.316813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:51.317176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] PATH_INFO: `/v3/projects/78ba0efcdbc24870b2ad9598dd3c0a89` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:51.319300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ba0efcdbc24870b2ad9598dd3c0a89)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:51.321512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:51.381937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ba0efcdbc24870b2ad9598dd3c0a89', 'request_id': 'req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.390907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ba0efcdbc24870b2ad9598dd3c0a89', 'request_id': 'req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.397317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ba0efcdbc24870b2ad9598dd3c0a89', 'request_id': 'req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.415259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] The token cache is being invalidate because project 78ba0efcdbc24870b2ad9598dd3c0a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:51.415513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.416262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.417836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1016/2041] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:51 2022] DELETE /identity/v3/projects/78ba0efcdbc24870b2ad9598dd3c0a89 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:51.423427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:51.435732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d470e9e-c451-4e01-9cb0-1049afedd4b5 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:51.504690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:51.505744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:51.506028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:51.506302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] PATH_INFO: `/v3/users/a0476a2f0fbb4e9abffd95414e35d02d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:51.507563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0476a2f0fbb4e9abffd95414e35d02d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:51.509530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:51.532291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:51.598004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0476a2f0fbb4e9abffd95414e35d02d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.611760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0476a2f0fbb4e9abffd95414e35d02d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.612507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0476a2f0fbb4e9abffd95414e35d02d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.657606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0476a2f0fbb4e9abffd95414e35d02d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.659934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-791f9bd0-5bb8-44b0-ab7d-f07263511b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0476a2f0fbb4e9abffd95414e35d02d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.671466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:51.675009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:51.698215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1026/2042] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:51 2022] DELETE /identity/v3/users/a0476a2f0fbb4e9abffd95414e35d02d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:51.704083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a2cfd-baef-4fe2-b795-d1a5fe7e79a7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:51.731511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:51.858436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:51.862956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:51.863270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:51.863555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] PATH_INFO: `/v3/users/648faa02a82b4186b284e926087be5c8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:51.868098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] RBAC: Authorizing `identity:delete_user(user_id=648faa02a82b4186b284e926087be5c8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:51.873132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:51.978839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648faa02a82b4186b284e926087be5c8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.991530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648faa02a82b4186b284e926087be5c8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:51.993268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648faa02a82b4186b284e926087be5c8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.019225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648faa02a82b4186b284e926087be5c8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.019944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648faa02a82b4186b284e926087be5c8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.045816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1017/2043] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:51 2022] DELETE /identity/v3/users/648faa02a82b4186b284e926087be5c8 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:52.052512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.053011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.053472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.080951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:52.082868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.188521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1027/2044] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:52.198114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.199748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.277453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.278885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.283998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6017c0-42a1-44bd-a895-32ef9e5a048c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.296192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65751d07-a3a0-41c0-b770-5a9cfb2a4ae5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.297333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.298610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.299933 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.300421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.316399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.318699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.320764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.322147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] PATH_INFO: `/v3/projects/6df08522cb4442c499616ff63ad85ea6` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.324427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df08522cb4442c499616ff63ad85ea6)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.328849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.331007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.334152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673040d5-b00e-422b-b834-08ca075485fe service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.350726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1018/2045] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:52.396215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df08522cb4442c499616ff63ad85ea6', 'request_id': 'req-b1e09957-5e29-42c1-bbc3-8310f86bdc80'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.406777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df08522cb4442c499616ff63ad85ea6', 'request_id': 'req-b1e09957-5e29-42c1-bbc3-8310f86bdc80'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.415822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df08522cb4442c499616ff63ad85ea6', 'request_id': 'req-b1e09957-5e29-42c1-bbc3-8310f86bdc80'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.428187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] The token cache is being invalidate because project 6df08522cb4442c499616ff63ad85ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:52.428768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.429514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b1e09957-5e29-42c1-bbc3-8310f86bdc80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:52.434916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1028/2046] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:52 2022] DELETE /identity/v3/projects/6df08522cb4442c499616ff63ad85ea6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:52.436673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.437020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.437409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.462062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:52.463396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.516474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:52.558205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1019/2047] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:52.571668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.573430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.574286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.614543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:52.616678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.632180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1029/2048] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:52.637159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.654292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc799a02-d9ec-47f8-adb8-1fcec28404eb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.663169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.664771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.665082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.665392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.667453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:52.668606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.672108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.696651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1020/2049] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:52.701438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.709287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:52.714343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b6bfe-45fc-45aa-82a3-e8f00be22255 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.723053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.723995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.724372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.724953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.725959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.728813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.772576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1030/2050] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:52 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:52.775143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.801711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a95e97-e0ec-4fe2-bfca-5e0d54224e74 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.903817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.907443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:52.914852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.915748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.916430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.916982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.917849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.920767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.942805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02deb331-48a8-46a6-a111-d2701041c6b0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.948400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1021/2051] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:52 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:52.949888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.951157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.951506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.951841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] PATH_INFO: `/v3/projects/695df26fb22c4241af9fd330ecd3db9e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.951997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:52.953413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] RBAC: Authorizing `identity:delete_project(project_id=695df26fb22c4241af9fd330ecd3db9e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.956713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.963754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a443f-b859-4a71-90f8-a60e2fb56236 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:52.969883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:52.970669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:52.970956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:52.971237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:52.971987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:52.973786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:52.986395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1022/2052] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:52.989659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.004535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39cc8b-0e78-423b-8eae-94f095ff97ba admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.010157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.011063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.011326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.011609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.012533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.014162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.028530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1023/2053] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:53.032405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695df26fb22c4241af9fd330ecd3db9e', 'request_id': 'req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.032567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.042823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695df26fb22c4241af9fd330ecd3db9e', 'request_id': 'req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.046485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08939273-e6e4-489b-92e6-5afae4e6acb4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.050496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695df26fb22c4241af9fd330ecd3db9e', 'request_id': 'req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.064228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] The token cache is being invalidate because project 695df26fb22c4241af9fd330ecd3db9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:53.064521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.065649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.067936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1031/2054] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:52 2022] DELETE /identity/v3/projects/695df26fb22c4241af9fd330ecd3db9e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:53.072137 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.073741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:53.118392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.119319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.119609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.119887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] PATH_INFO: `/v3/projects/06e61952425f4918bcd0cd5249a63b19/users/2b25f7cc6c3f4c918925725f1a908d41/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.136174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e61952425f4918bcd0cd5249a63b19, user_id=2b25f7cc6c3f4c918925725f1a908d41, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.139020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db119f53-13ac-4597-b1f1-070d309db86b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.148771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec135f-7e9e-4ea4-8740-5eff2a4f282b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.162522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.163244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.163395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1024/2055] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:53 2022] PUT /identity/v3/projects/06e61952425f4918bcd0cd5249a63b19/users/2b25f7cc6c3f4c918925725f1a908d41/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:53.164109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.164524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] PATH_INFO: `/v3/users/7d84222dffc44024923534abd40e0a21` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.166582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d84222dffc44024923534abd40e0a21)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.168573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.174273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.174836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.175042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.221285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] MFA Rules not processed for user `2b25f7cc6c3f4c918925725f1a908d41`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:53.222817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:53.229220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:53.244781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d84222dffc44024923534abd40e0a21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.276838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d84222dffc44024923534abd40e0a21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.278759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d84222dffc44024923534abd40e0a21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.294528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:53.297958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d84222dffc44024923534abd40e0a21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.299603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d84222dffc44024923534abd40e0a21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:53.319548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1025/2056] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:53.321589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1032/2057] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:53 2022] DELETE /identity/v3/users/7d84222dffc44024923534abd40e0a21 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:53.326596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.328050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:53.407352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2192145-1808-4651-bbd2-6f3f9217a351 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.471721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.472402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.472686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.472957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.473676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.475664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.506271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1026/2058] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:53 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:53.510660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.522158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85733-1f06-430f-8dd7-e842d43b45ad admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.585351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.586018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.586297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.586599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.587306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.589291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.618303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1033/2059] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:53 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:53.619804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.631238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e6a0b-571e-4fac-941c-e22a8be7e756 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.636119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.636798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.637083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.637357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.638095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.639754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.654577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1027/2060] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:53.655873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.667811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc1300-21c4-4766-8543-8f9e11929cb5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.676751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.677577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.677954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.678213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.679053 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.680693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfabe0f-6ca5-4b94-afa7-f75e908e2fea admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.700398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1034/2061] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:53.704661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.715461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1258a-d90c-4ddb-8aeb-a9966ee9fd36 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.720328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.721233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.721512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.721786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] PATH_INFO: `/v3/projects/d9e6a87f876a4c41977382ef9ac05477/users/62fcc181b57a4cb3a21b7065b6b54344/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.739327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9e6a87f876a4c41977382ef9ac05477, user_id=62fcc181b57a4cb3a21b7065b6b54344, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.742256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.758635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1028/2062] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:53 2022] PUT /identity/v3/projects/d9e6a87f876a4c41977382ef9ac05477/users/62fcc181b57a4cb3a21b7065b6b54344/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:53.771154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.771541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.771825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.815823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] MFA Rules not processed for user `62fcc181b57a4cb3a21b7065b6b54344`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:53.817349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:53.865067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.867469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:53.873219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:53.900778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1035/2063] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:53.907023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:53.952427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49453ee4-96e8-48e8-87f7-24fcd94db7f1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.959101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.960549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.960902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.961352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] PATH_INFO: `/v3/projects/10773d86a70b41eba14c3faaf10b4871` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.962838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10773d86a70b41eba14c3faaf10b4871)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.965190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:53.968286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:53.981163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04a38-6321-42f4-bc8e-1177a7aed54e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:53.987474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:53.988226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:53.988513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:53.988804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:53.989640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:53.991779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.015619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10773d86a70b41eba14c3faaf10b4871', 'request_id': 'req-f9b394de-a2c9-403f-9ec6-3494b1e6a347'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.019692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1036/2064] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:53 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:54.023950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.027041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10773d86a70b41eba14c3faaf10b4871', 'request_id': 'req-f9b394de-a2c9-403f-9ec6-3494b1e6a347'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.033995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10773d86a70b41eba14c3faaf10b4871', 'request_id': 'req-f9b394de-a2c9-403f-9ec6-3494b1e6a347'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.049397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] The token cache is being invalidate because project 10773d86a70b41eba14c3faaf10b4871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:54.049701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.050477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.052224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1029/2065] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:53 2022] DELETE /identity/v3/projects/10773d86a70b41eba14c3faaf10b4871 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:54.056680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.058377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.091910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af13aa-7dc7-47da-b9bf-ec706b56bdd8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.096846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.097596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.097953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.098292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.099128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.101145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.126418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1037/2066] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:54.128106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.130276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.133491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b394de-a2c9-403f-9ec6-3494b1e6a347 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.138180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.139132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.139406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.139674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] PATH_INFO: `/v3/users/7483b38fdcf4493986c2efd6a91be278` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.140873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7483b38fdcf4493986c2efd6a91be278)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.142880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.143009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692b820-4d2f-47bc-906f-b312705812c3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.147860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.148516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.148797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.149065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.149795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.151432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.161588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1038/2067] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:54.165862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.178885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7112a018-65e6-4edf-8902-0a11f1af8842 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.183809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.184745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.185003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.185272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] PATH_INFO: `/v3/projects/b246af6464284cfd8c6f388ce3ea588b/users/1a9f1afc01854418abb3be9ef191d61f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.190199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:54.199883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7483b38fdcf4493986c2efd6a91be278'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.202301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b246af6464284cfd8c6f388ce3ea588b, user_id=1a9f1afc01854418abb3be9ef191d61f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.205376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.212155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7483b38fdcf4493986c2efd6a91be278'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.212902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7483b38fdcf4493986c2efd6a91be278'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.221475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1039/2068] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:54 2022] PUT /identity/v3/projects/b246af6464284cfd8c6f388ce3ea588b/users/1a9f1afc01854418abb3be9ef191d61f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:54.225564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.226434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7483b38fdcf4493986c2efd6a91be278'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.227138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7483b38fdcf4493986c2efd6a91be278'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.227326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.239821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1030/2069] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:54 2022] DELETE /identity/v3/users/7483b38fdcf4493986c2efd6a91be278 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:54.303713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21418bdc-972b-4eae-a686-aa493078eed6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.346687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:54.368615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.369286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.369565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.369837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.370606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.372221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca9dc9-fd16-4792-87c8-9bb7111e49f1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.383349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1040/2070] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:54.387829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.412918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b31183-7630-4f80-9555-1168fd921985 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.422611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.422611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.422611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.422611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] PATH_INFO: `/v3/projects/b246af6464284cfd8c6f388ce3ea588b/users/1a9f1afc01854418abb3be9ef191d61f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.435867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b246af6464284cfd8c6f388ce3ea588b, user_id=1a9f1afc01854418abb3be9ef191d61f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.438617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.457300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1031/2071] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:54 2022] PUT /identity/v3/projects/b246af6464284cfd8c6f388ce3ea588b/users/1a9f1afc01854418abb3be9ef191d61f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:54.467964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.468254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.468824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.502455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] MFA Rules not processed for user `1a9f1afc01854418abb3be9ef191d61f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:54.503740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.531681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.533439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.586831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1041/2072] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:54.599579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.600966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.606175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c461898-4e05-4bb4-93f2-6109d0ec940f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.611020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.611907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.612194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.612473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] PATH_INFO: `/v3/projects/75a1596d9a5a4094bbd72004de9789a1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.613654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a1596d9a5a4094bbd72004de9789a1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.615849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.660156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde1bdb-22c4-40a4-b72c-c29f60ed898c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.663544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a1596d9a5a4094bbd72004de9789a1', 'request_id': 'req-d74d345c-26d0-49f3-8b42-6bed45b130b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.672127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a1596d9a5a4094bbd72004de9789a1', 'request_id': 'req-d74d345c-26d0-49f3-8b42-6bed45b130b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.678893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a1596d9a5a4094bbd72004de9789a1', 'request_id': 'req-d74d345c-26d0-49f3-8b42-6bed45b130b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.689608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] The token cache is being invalidate because project 75a1596d9a5a4094bbd72004de9789a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:54.689827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.690540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d74d345c-26d0-49f3-8b42-6bed45b130b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:54.692145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1032/2073] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:54 2022] DELETE /identity/v3/projects/75a1596d9a5a4094bbd72004de9789a1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:54.697736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0bd-4d66-4d53-88ba-93031e57f443 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.698027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0bd-4d66-4d53-88ba-93031e57f443 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.698309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82daa0bd-4d66-4d53-88ba-93031e57f443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.714411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.715400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.715680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.715954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.717051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-82daa0bd-4d66-4d53-88ba-93031e57f443 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:54.718071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.718246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-82daa0bd-4d66-4d53-88ba-93031e57f443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.746602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:54.755587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:54.786436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1033/2074] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:54.793620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.795231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.795382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.795520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.795811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.805566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1042/2075] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:54.813597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:54.814820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.824409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1034/2076] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:54.828893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.841392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777356b-ff27-4180-bc7f-4ef179b34176 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.846542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.847430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.847620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.847878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.848775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.848893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:54.849171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.850539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.850844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.866357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1043/2077] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:54.866534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35714dba-41b7-460b-9f5d-1bbc065b5ad3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.867860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.869515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.870411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.870695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.870970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.872805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:54.880207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197e30b-9f7e-4be2-8dcd-a74404cd719a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:54.885154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:54.885872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:54.886145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:54.886521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:54.887266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.889199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.922749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1044/2078] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:54.924429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:54.940445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:54.941869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:54.951722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1035/2079] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:55.003757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7edf28-22e0-428a-8b1c-ff8d17336bb6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.008374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.009049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.009348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.009609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.010327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.012310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.033202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1045/2080] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:54 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:55.038541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.049371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f648519-214b-4af3-b2fe-167232390dd9 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.054717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.055499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.055783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.056051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.056827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.058560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.069722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1036/2081] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:55.073947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.097748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69e6d8-f3a7-4a73-b117-e0f934c2f6e0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.102720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.103395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.103675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.103950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.104687 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.106357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce965dea-0756-4f8d-8860-8b0e69f83593 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.117352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1046/2082] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:55.121636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.133265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ed27-7b3b-42ed-9fef-c0091502d5c7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.139932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.141556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.142255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.142634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] PATH_INFO: `/v3/projects/927dc67ba0dc426d8753e2510fd3c7ee/users/64369751380d4c94b682a2234d1af16b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.166621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927dc67ba0dc426d8753e2510fd3c7ee, user_id=64369751380d4c94b682a2234d1af16b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.169397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.187247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1037/2083] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:55 2022] PUT /identity/v3/projects/927dc67ba0dc426d8753e2510fd3c7ee/users/64369751380d4c94b682a2234d1af16b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:55.199493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.199691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.200013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.229793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] MFA Rules not processed for user `64369751380d4c94b682a2234d1af16b`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:55.231048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:55.309407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:55.318402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1047/2084] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:55.327743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.330015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:55.392201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7cb87-d204-460e-9794-9db296ee8079 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.397066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.397725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.398015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.398266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.408320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.409838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0b38d-c38b-416e-9fd7-693257965d73 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.418849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1038/2085] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:55.592008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.593638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:55.664554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc8b72-07a2-413c-ab3c-2e58a9eb736e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.670809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.672088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.672288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.672706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] PATH_INFO: `/v3/users/82703d20c3834d148ce525b17a98367f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.674094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82703d20c3834d148ce525b17a98367f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.676627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.745763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82703d20c3834d148ce525b17a98367f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:55.755959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82703d20c3834d148ce525b17a98367f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:55.756597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82703d20c3834d148ce525b17a98367f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:55.774907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82703d20c3834d148ce525b17a98367f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:55.775738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82703d20c3834d148ce525b17a98367f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:55.789810 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1048/2086] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:55 2022] DELETE /identity/v3/users/82703d20c3834d148ce525b17a98367f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:55.794475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.794843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.795188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.825731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:55.827006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:55.854137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:55.890816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1039/2087] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:55.901064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:55.902585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:55.962270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236ae31-3c23-4332-95f2-b93eb43e76d4 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:55.967051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:55.967848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:55.968136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:55.968420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:55.978741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:55.980184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:55.989993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1049/2088] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:56.288481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:56.290074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:56.302912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59967338-6fdd-4a59-852e-9e836e8d9c7e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:56.309519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:56.310653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:56.310871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:56.311275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] PATH_INFO: `/v3/projects/9f54a08332aa4a5d8ccb9e53c76158ea` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:56.312655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f54a08332aa4a5d8ccb9e53c76158ea)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:56.315394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:56.385126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f54a08332aa4a5d8ccb9e53c76158ea', 'request_id': 'req-b0b31365-628f-40ae-91d4-08c20fdfae2f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.398837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f54a08332aa4a5d8ccb9e53c76158ea', 'request_id': 'req-b0b31365-628f-40ae-91d4-08c20fdfae2f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.407666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f54a08332aa4a5d8ccb9e53c76158ea', 'request_id': 'req-b0b31365-628f-40ae-91d4-08c20fdfae2f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.424450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] The token cache is being invalidate because project 9f54a08332aa4a5d8ccb9e53c76158ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:56.424753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.425411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.428700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1040/2089] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:56 2022] DELETE /identity/v3/projects/9f54a08332aa4a5d8ccb9e53c76158ea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:56.432615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:56.444212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67e2ae-edb1-44d5-98a2-8efaf3d5d620 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:56.476865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:56.520128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:56.521419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:56.521770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:56.522111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] PATH_INFO: `/v3/users/2b2dac7de96f4b8b9f49c83f70354725` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:56.523251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2dac7de96f4b8b9f49c83f70354725)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:56.525188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:56.577205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2dac7de96f4b8b9f49c83f70354725'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.587796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2dac7de96f4b8b9f49c83f70354725'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.588504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2dac7de96f4b8b9f49c83f70354725'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.601897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2dac7de96f4b8b9f49c83f70354725'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.602615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cf135ee3-dd78-435c-a8dc-7877e251f080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2dac7de96f4b8b9f49c83f70354725'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:56.615538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1050/2090] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:56 2022] DELETE /identity/v3/users/2b2dac7de96f4b8b9f49c83f70354725 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:56.911827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:56.913260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:56.928972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b31365-628f-40ae-91d4-08c20fdfae2f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:56.986114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:56.986926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:56.987208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:56.987467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] PATH_INFO: `/v3/projects/b456a528c4b44d9ab152b91c72ff5143` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:56.988635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b456a528c4b44d9ab152b91c72ff5143)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:56.990798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.038916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b456a528c4b44d9ab152b91c72ff5143', 'request_id': 'req-f3ea6584-d995-4505-8a5c-6c3753502365'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.048837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b456a528c4b44d9ab152b91c72ff5143', 'request_id': 'req-f3ea6584-d995-4505-8a5c-6c3753502365'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.050199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:57.058540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b456a528c4b44d9ab152b91c72ff5143', 'request_id': 'req-f3ea6584-d995-4505-8a5c-6c3753502365'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.071250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] The token cache is being invalidate because project b456a528c4b44d9ab152b91c72ff5143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:57.071541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.072245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f3ea6584-d995-4505-8a5c-6c3753502365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.073888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1041/2091] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:56 2022] DELETE /identity/v3/projects/b456a528c4b44d9ab152b91c72ff5143 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:57.081904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.082199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.082512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.101913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:57.103184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:57.163314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1051/2092] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:57.171814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.172102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.172377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.192977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:57.194222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:57.203445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1042/2093] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:57.207990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.218372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25de4f-ff09-4139-ba3b-91d52c668f6e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.223636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.224324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.224602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.224875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.225588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:57.225907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.227699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.239539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1052/2094] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:57.244006 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.253412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77328c-91d1-48e8-a378-0566c1f5c59e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.259288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.259956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.260235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.260506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.261199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.263204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.296016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1043/2095] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:06:57.301368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.311881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65a985-55fc-4744-922a-18f8145deb2c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.340738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:57.376735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.377541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.377825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.378107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.378871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.381251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.404896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1053/2096] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:57.409461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.420672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a97eae-6b39-42ff-9b57-df04d088336d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.425798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.426587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.426867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.427109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.427857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.429459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.440472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1044/2097] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:57.444829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.455540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd7c2-43e6-4f03-89ff-0454518989fe admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.461054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.461823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.462138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.462416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.463159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.465486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36d04c-8fb5-4c7c-8285-c7e451631468 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.478165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1054/2098] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:06:57.482438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.493269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841727d6-247e-488a-ab39-333fad5ad70c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.498715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.499607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.499889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.500163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] PATH_INFO: `/v3/projects/79e4265b7ddc455daf9e47fe7b42f1bd/users/0e4ddadf5f23493ba804f8b26817fb21/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.516204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e4265b7ddc455daf9e47fe7b42f1bd, user_id=0e4ddadf5f23493ba804f8b26817fb21, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.519272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.537099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1045/2099] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:57 2022] PUT /identity/v3/projects/79e4265b7ddc455daf9e47fe7b42f1bd/users/0e4ddadf5f23493ba804f8b26817fb21/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:57.548767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.549062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.549372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.580434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] MFA Rules not processed for user `0e4ddadf5f23493ba804f8b26817fb21`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:57.581673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:57.649161 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1055/2100] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:57.660693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.662777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:57.722159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9c134-9d4f-455f-a96d-db33fd5a6469 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.727118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.728053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.728356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.728626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.741857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.743442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.752738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1046/2101] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:57.773312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.844418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae11d4-9b5d-465c-b076-c320c5b596a6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.849914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:57.854827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.855250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.855550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] PATH_INFO: `/v3/users/0e4ddadf5f23493ba804f8b26817fb21` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.856875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4ddadf5f23493ba804f8b26817fb21)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:57.858905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:57.911819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:57.919657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4ddadf5f23493ba804f8b26817fb21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.931215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4ddadf5f23493ba804f8b26817fb21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.931897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4ddadf5f23493ba804f8b26817fb21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.950842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4ddadf5f23493ba804f8b26817fb21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.951612 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a0099711-2640-40ab-9b6e-b1ebb38db371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4ddadf5f23493ba804f8b26817fb21'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:57.967036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:57.968770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:57.969883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1056/2102] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:57 2022] DELETE /identity/v3/users/0e4ddadf5f23493ba804f8b26817fb21 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:57.974512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:57.974873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:57.975293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:57.987175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be9e8b-558f-412c-8146-c10a3f668b81 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:57.997308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:57.998650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.059457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.061032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.061322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.061780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] PATH_INFO: `/v3/users/2b25f7cc6c3f4c918925725f1a908d41` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.062865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b25f7cc6c3f4c918925725f1a908d41)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.065094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.077817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1057/2103] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:58.090551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.091953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.105061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:58.127206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b25f7cc6c3f4c918925725f1a908d41'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.136673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b25f7cc6c3f4c918925725f1a908d41'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.137418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b25f7cc6c3f4c918925725f1a908d41'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.150004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b25f7cc6c3f4c918925725f1a908d41'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.150666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-39cd4683-9732-411b-81f0-51d77ddd72f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b25f7cc6c3f4c918925725f1a908d41'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.155089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cc635-d779-4156-9daf-fa8855d5d9c7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.158897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.159539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.159816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.160087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.162622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.162921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1047/2104] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:57 2022] DELETE /identity/v3/users/2b25f7cc6c3f4c918925725f1a908d41 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:58.167362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.167655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.167931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.175791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.177394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.185662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:58.187008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1058/2105] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:06:58.187341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.253398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1048/2106] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:58.264055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.265690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.277462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f197c2-64e4-44fe-a274-ab8731417567 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.338546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.339205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.339490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.340111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.350290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.351826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a46be-a457-4ed3-88a9-24ae3e11455b service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.363202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1059/2107] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:58.509183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.510744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.529847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856f422-c04c-4cbc-b562-fcc4a7da7b2a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.535288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.536058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.537035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.537381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] PATH_INFO: `/v3/projects/79e4265b7ddc455daf9e47fe7b42f1bd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.538701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e4265b7ddc455daf9e47fe7b42f1bd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.540667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.595594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e4265b7ddc455daf9e47fe7b42f1bd', 'request_id': 'req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.606203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e4265b7ddc455daf9e47fe7b42f1bd', 'request_id': 'req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.613746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e4265b7ddc455daf9e47fe7b42f1bd', 'request_id': 'req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.624442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] The token cache is being invalidate because project 79e4265b7ddc455daf9e47fe7b42f1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:58.624725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.625376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.626952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1049/2108] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:58 2022] DELETE /identity/v3/projects/79e4265b7ddc455daf9e47fe7b42f1bd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:58.633154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ef10-d666-489d-abb4-a8ba034dd30e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.633563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ef10-d666-489d-abb4-a8ba034dd30e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.633937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6ef10-d666-489d-abb4-a8ba034dd30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.642671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:58.659409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9fe6ef10-d666-489d-abb4-a8ba034dd30e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:58.661546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9fe6ef10-d666-489d-abb4-a8ba034dd30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.666353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.668637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.710016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:58.737614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1060/2109] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:58.754638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fef10-65cf-4c5c-81cd-8b7ae62c02ce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.762260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.763235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.763534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.763812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] PATH_INFO: `/v3/projects/06e61952425f4918bcd0cd5249a63b19` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.765375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e61952425f4918bcd0cd5249a63b19)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.767663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.811256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.811623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.811836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.825322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e61952425f4918bcd0cd5249a63b19', 'request_id': 'req-e203afb8-0db3-486e-9073-6606b403698f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.833604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:58.835453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.839259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e61952425f4918bcd0cd5249a63b19', 'request_id': 'req-e203afb8-0db3-486e-9073-6606b403698f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.848995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e61952425f4918bcd0cd5249a63b19', 'request_id': 'req-e203afb8-0db3-486e-9073-6606b403698f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.860732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] The token cache is being invalidate because project 06e61952425f4918bcd0cd5249a63b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:58.861019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.861704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:58.863289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1050/2110] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:58 2022] DELETE /identity/v3/projects/06e61952425f4918bcd0cd5249a63b19 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:58.868357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.871124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:58.906415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1061/2111] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:58.911083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.920948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5f45e-962b-48d1-bc31-bf853389197d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.926182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.927384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.927527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.927805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.928546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:06:58.928854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.930533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.937830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203afb8-0db3-486e-9073-6606b403698f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.942751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.943370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1062/2112] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:06:58.943567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.943854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.944109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] PATH_INFO: `/v3/users/62fcc181b57a4cb3a21b7065b6b54344` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.945313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fcc181b57a4cb3a21b7065b6b54344)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.947275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:58.947759 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:58.957347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d891b-6b25-4b2a-9d9b-23a91b0c5f1f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:58.965648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:58.966501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:58.966806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:58.967112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:58.967814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:58.970023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.006217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1063/2113] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:59.010431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.013156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fcc181b57a4cb3a21b7065b6b54344'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.025308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fcc181b57a4cb3a21b7065b6b54344'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.026777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fcc181b57a4cb3a21b7065b6b54344'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.033264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:59.055713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fcc181b57a4cb3a21b7065b6b54344'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.056877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fcc181b57a4cb3a21b7065b6b54344'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.072244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1051/2114] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:58 2022] DELETE /identity/v3/users/62fcc181b57a4cb3a21b7065b6b54344 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:59.080467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982d567-887f-4e8f-b5a4-9855a561bf9c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.092697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:59.134765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.136045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.148732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.149595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.150074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.150441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.151595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.153607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.180170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1064/2115] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:06:59 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:06:59.195286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.203639 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.220614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa1972-1a53-4bbb-917c-cd4984f8670e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.226028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f445fb5-be3d-48b3-87a6-56510e88e599 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.226613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.227588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.227934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.228269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] PATH_INFO: `/v3/users/64369751380d4c94b682a2234d1af16b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.229698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64369751380d4c94b682a2234d1af16b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.231802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.235561 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.236383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.236779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.237112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.238005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.241291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.268675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1065/2116] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:59.276023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.288357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95193a34-c592-4a6d-b9b9-79ab3ddfd26d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.295726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.295726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.296499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.296782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.297519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.299478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.315007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64369751380d4c94b682a2234d1af16b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.318495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1066/2117] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:06:59.320261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.332129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64369751380d4c94b682a2234d1af16b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.332129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64369751380d4c94b682a2234d1af16b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.338224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7047af6-fc20-4484-8b56-80259c3e8946 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.345663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.347070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.347491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.347903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] PATH_INFO: `/v3/projects/e2cf537d534d4e7fae7f415416dadb52/users/268a7c0d30b14fd9ac567cf6f60fcffc/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.354813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64369751380d4c94b682a2234d1af16b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.355587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0c08c4ae-6603-49e2-b42c-3977d56bf35f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64369751380d4c94b682a2234d1af16b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.375739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2cf537d534d4e7fae7f415416dadb52, user_id=268a7c0d30b14fd9ac567cf6f60fcffc, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.378806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fc33d-71ba-4c93-b387-d5fe8a24a6e3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.383509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1052/2118] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:59 2022] DELETE /identity/v3/users/64369751380d4c94b682a2234d1af16b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:06:59.387851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.388142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.388453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.418726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1067/2119] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:06:59 2022] PUT /identity/v3/projects/e2cf537d534d4e7fae7f415416dadb52/users/268a7c0d30b14fd9ac567cf6f60fcffc/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:59.419834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:59.420946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.421376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.421793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.423872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.470130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] MFA Rules not processed for user `268a7c0d30b14fd9ac567cf6f60fcffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:06:59.471514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.498566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:59.514138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1053/2120] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:06:59.516460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.518618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.533980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff5f29-8d69-4428-a065-8d07a103a02e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.539241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.540116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.540396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.540670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] PATH_INFO: `/v3/projects/d9e6a87f876a4c41977382ef9ac05477` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.542029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9e6a87f876a4c41977382ef9ac05477)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.543990 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.566823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1068/2121] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:06:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:06:59.566823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.566823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.606141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9e6a87f876a4c41977382ef9ac05477', 'request_id': 'req-15abd987-6583-409f-8873-9e64e895a45d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.616808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9e6a87f876a4c41977382ef9ac05477', 'request_id': 'req-15abd987-6583-409f-8873-9e64e895a45d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.628227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9e6a87f876a4c41977382ef9ac05477', 'request_id': 'req-15abd987-6583-409f-8873-9e64e895a45d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.636119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:06:59.640534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d14dc-adcd-4720-a13b-9a04bb431b61 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.640840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] The token cache is being invalidate because project d9e6a87f876a4c41977382ef9ac05477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:06:59.641304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.642253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.644333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1054/2122] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:06:59 2022] DELETE /identity/v3/projects/d9e6a87f876a4c41977382ef9ac05477 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:06:59.645407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.648191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.712199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.712862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.713138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.713411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.715710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.725206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abd987-6583-409f-8873-9e64e895a45d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.729863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.731148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.731562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.732556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.739738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.800520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.802431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.815409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1069/2123] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:59.816508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:06:59.818405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:06:59.837093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb887a6a-983d-451f-893a-e951f788888c service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:06:59.844226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:06:59.845516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:06:59.845933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:06:59.846305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] PATH_INFO: `/v3/users/1a9f1afc01854418abb3be9ef191d61f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:06:59.847763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9f1afc01854418abb3be9ef191d61f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.850533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.879847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:06:59.881785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:06:59.900070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1055/2124] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:06:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:06:59.957254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9f1afc01854418abb3be9ef191d61f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.976552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9f1afc01854418abb3be9ef191d61f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.979851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9f1afc01854418abb3be9ef191d61f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.998763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9f1afc01854418abb3be9ef191d61f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:06:59.999559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9f1afc01854418abb3be9ef191d61f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.017351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1070/2125] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:06:59 2022] DELETE /identity/v3/users/1a9f1afc01854418abb3be9ef191d61f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:00.394757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.396335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.431574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a9a81-5514-41fd-990d-57c3fff97895 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.476206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:00.505627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.508241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.529568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.530565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.530784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.531070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] PATH_INFO: `/v3/projects/927dc67ba0dc426d8753e2510fd3c7ee` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.532817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927dc67ba0dc426d8753e2510fd3c7ee)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.534537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.535335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8d25e-14f5-46bb-8923-a41f7eec7546 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.541730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.542948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.543764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.543972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] PATH_INFO: `/v3/projects/b246af6464284cfd8c6f388ce3ea588b` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.545361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b246af6464284cfd8c6f388ce3ea588b)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.547567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.573663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:00.586926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927dc67ba0dc426d8753e2510fd3c7ee', 'request_id': 'req-b9e179e7-109f-4df4-b363-68cbc532eff9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.598911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927dc67ba0dc426d8753e2510fd3c7ee', 'request_id': 'req-b9e179e7-109f-4df4-b363-68cbc532eff9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.605680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927dc67ba0dc426d8753e2510fd3c7ee', 'request_id': 'req-b9e179e7-109f-4df4-b363-68cbc532eff9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.606129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b246af6464284cfd8c6f388ce3ea588b', 'request_id': 'req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.616716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b246af6464284cfd8c6f388ce3ea588b', 'request_id': 'req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.620653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] The token cache is being invalidate because project 927dc67ba0dc426d8753e2510fd3c7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:00.620976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.621745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b9e179e7-109f-4df4-b363-68cbc532eff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.623494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1056/2126] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:00 2022] DELETE /identity/v3/projects/927dc67ba0dc426d8753e2510fd3c7ee => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:00.624470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b246af6464284cfd8c6f388ce3ea588b', 'request_id': 'req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.628242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ea91-c96c-4916-b9de-d86ebd703f08 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.628602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ea91-c96c-4916-b9de-d86ebd703f08 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.628949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2ea91-c96c-4916-b9de-d86ebd703f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.636849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] The token cache is being invalidate because project b246af6464284cfd8c6f388ce3ea588b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:00.636849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.637070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c9d1d1f4-58c5-4561-8cee-6b0ab3f9455c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:00.638567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1071/2127] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:00 2022] DELETE /identity/v3/projects/b246af6464284cfd8c6f388ce3ea588b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:07:00.644640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fa91-400a-4074-a882-af19bba020bd None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.645010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fa91-400a-4074-a882-af19bba020bd None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.645427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215fa91-400a-4074-a882-af19bba020bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.647377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-34e2ea91-c96c-4916-b9de-d86ebd703f08 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:00.648590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-34e2ea91-c96c-4916-b9de-d86ebd703f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.664065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8215fa91-400a-4074-a882-af19bba020bd None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:00.665256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8215fa91-400a-4074-a882-af19bba020bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.708950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1057/2128] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:07:00.716962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.717252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.717532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.729364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1072/2129] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:07:00.738242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:00.738578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.738875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.739426 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.739603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.748535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1058/2130] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:00.752637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.758209 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:00.759541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:00.762790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55ed3d-f1f0-4b3e-8968-ad1b8688f044 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.767855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1073/2131] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:00.768937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.769706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.769987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.770393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.771461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:07:00.771776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.772739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.773417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.782808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021d21b-41d8-46db-9525-929ec0262daf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.787984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.788134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1059/2132] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:07:00.788655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.788941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.789220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.789955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:07:00.790273 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.791947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.792318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.802891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8144d68-d406-42cf-97a1-af9b81e69f35 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.807883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1074/2133] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:07:00.808283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.808965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.809245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.809435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.809621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.810356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.812366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.819703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2203bf6-13df-4367-9469-0e2432314405 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.825726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.826609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.826891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.827172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.827876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.830021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.844428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1060/2134] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:07:00.845893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.863026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1075/2135] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:07:00.867207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.871239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:00.915156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b444bd-a17b-470e-8bfa-cb5dc24e1a91 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.936378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b9848-eb52-4e64-bfd7-73e91ec35e28 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.941743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.942695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.942977 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.943251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.943982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.946040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.965084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:00.975656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1076/2136] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:07:00.976583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:00.985626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.986566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.986709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.987029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.987724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:00.988744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82afc3-13a2-4b38-afb4-9a9b8fe9fa3d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:00.990588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:00.995454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:00.996682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:00.997205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:00.997582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:00.998406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.000101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.013363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1077/2137] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:01.014402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1061/2138] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:07:00 2022] POST /identity/v3/users => generated 282 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:07:01.017440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.018356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.029912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baefe413-b4aa-4f02-87f4-3c57abbd0c42 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.030146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d1b7e-fb09-46c9-a995-c9f247628ef2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.036086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.036776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.037060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.037335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.038092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.038990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.039678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.039863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.039975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.040228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.040970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.042614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.058487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1078/2139] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:01.062234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.063018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1062/2140] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:07:01.065996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.078261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dfeda-498e-47f7-8e7f-efa27a78397f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.079441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1c6b6a-9052-4c60-a4d1-e1d5c80a9d93 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.084864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.085762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.085965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.086079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.086312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] PATH_INFO: `/v3/projects/1284e356314c4a4da786c6bb14650132/users/00696cd5d92e459ca2650f3a91abdfd8/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.086799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.087084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.087602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.088398 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.092308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.110974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1284e356314c4a4da786c6bb14650132, user_id=00696cd5d92e459ca2650f3a91abdfd8, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.111222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1063/2141] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:01.111321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d48d4-6b7b-454c-b464-14b390aa5ac6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.112298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.124778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f9b0a-02df-4eff-abce-32323236086d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.138067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1079/2142] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:07:01 2022] PUT /identity/v3/projects/1284e356314c4a4da786c6bb14650132/users/00696cd5d92e459ca2650f3a91abdfd8/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:01.150362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.151401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.151933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.189337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] MFA Rules not processed for user `00696cd5d92e459ca2650f3a91abdfd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:01.190809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:01.226250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.227218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.227499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.227764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] PATH_INFO: `/v3/projects/7246586d5ea74352a1676308edab1fe8/users/6cb44f0155eb423a9330a71410f3461d/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.245046 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7246586d5ea74352a1676308edab1fe8, user_id=6cb44f0155eb423a9330a71410f3461d, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.247812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba870ac5-cdd3-4beb-bb32-d378424a2c7a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.262580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1064/2143] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:07:01 2022] PUT /identity/v3/projects/7246586d5ea74352a1676308edab1fe8/users/6cb44f0155eb423a9330a71410f3461d/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:01.273077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1080/2144] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:07:01.275916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.276339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.276570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.280533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.282543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:01.305301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] MFA Rules not processed for user `6cb44f0155eb423a9330a71410f3461d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:01.306473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:01.314413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:01.342547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214a05-1b72-430d-8fff-f323bfb367f3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.379891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1065/2145] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:01.397766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.398452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.398724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.398995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.409501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:01.429967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:01.431698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:01.446250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f674-0cb4-4b3d-8c07-373afacbb847 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:01.451118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:01.451118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:01.451693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:01.451979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:01.462108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.463609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.470771 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:01.472217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb89a9c-2dde-4d2f-8d97-9975bdf75e80 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:01.473842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1066/2146] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:07:01.481963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1081/2147] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:07:11.469537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=29 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:11.477529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=22 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:12.001579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:12.108982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188828e1-784e-48d0-a397-fbc32d923196 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:12.114101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:12.115075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:12.115396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:12.115667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] PATH_INFO: `/v3/users/00696cd5d92e459ca2650f3a91abdfd8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:12.116887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00696cd5d92e459ca2650f3a91abdfd8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:12.119065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:12.182067 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00696cd5d92e459ca2650f3a91abdfd8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.193925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00696cd5d92e459ca2650f3a91abdfd8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.194621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00696cd5d92e459ca2650f3a91abdfd8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.211395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00696cd5d92e459ca2650f3a91abdfd8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.213591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00696cd5d92e459ca2650f3a91abdfd8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.240204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1067/2148] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:07:11 2022] DELETE /identity/v3/users/00696cd5d92e459ca2650f3a91abdfd8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:12.251726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:12.252602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:12.252602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:12.286158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:12.287486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:12.378788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1082/2149] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:12.389417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:12.390901 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:12.455854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b377fe-7461-496b-a5c2-00eb273a0507 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:12.459593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:12.460294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:12.460603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:12.460887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:12.471986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:12.474536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd9b7d-dcc3-4090-bd48-c29a0303545e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:12.485728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1068/2150] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:12.797311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:12.798871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:12.818006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd103be-daca-4dc3-b275-a8f3d889a51b None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:12.825950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:12.827170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:12.827544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:12.827880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] PATH_INFO: `/v3/projects/1284e356314c4a4da786c6bb14650132` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:12.829649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1284e356314c4a4da786c6bb14650132)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:12.832305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:12.892217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1284e356314c4a4da786c6bb14650132', 'request_id': 'req-c5769e81-8421-4166-b101-080f8f06c602'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.902959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1284e356314c4a4da786c6bb14650132', 'request_id': 'req-c5769e81-8421-4166-b101-080f8f06c602'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.910877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1284e356314c4a4da786c6bb14650132', 'request_id': 'req-c5769e81-8421-4166-b101-080f8f06c602'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.923177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] The token cache is being invalidate because project 1284e356314c4a4da786c6bb14650132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:12.923628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.924364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5769e81-8421-4166-b101-080f8f06c602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:12.926783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1083/2151] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:12 2022] DELETE /identity/v3/projects/1284e356314c4a4da786c6bb14650132 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:12.931756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:12.932056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:12.932917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:12.953037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:12.954375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:12.974944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:13.022810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1069/2152] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:13.031196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.031727 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.032007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.055566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:13.056652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.065729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1084/2153] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:13.070275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.080370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.082098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.084575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a453fa-ec5b-4801-926d-55f3a4ebdc15 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.096968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dabaa-acab-4a86-8516-6cf429d9fbce None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.100450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.101141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.101422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.101695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.102809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:07:13.102961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.103797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.104763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.104967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.105733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.106010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] PATH_INFO: `/v3/users/268a7c0d30b14fd9ac567cf6f60fcffc` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.107348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=268a7c0d30b14fd9ac567cf6f60fcffc)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.110781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.148799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1070/2154] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:07:13.153590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.165841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27631706-f9d7-4d29-a044-fd8d22a0f134 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.171695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.172447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.172796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.174359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.175288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.177753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.199326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268a7c0d30b14fd9ac567cf6f60fcffc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:13.212409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268a7c0d30b14fd9ac567cf6f60fcffc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:13.216640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268a7c0d30b14fd9ac567cf6f60fcffc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:13.229587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1071/2155] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:07:13 2022] POST /identity/v3/projects => generated 399 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:07:13.231323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.233047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.235828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268a7c0d30b14fd9ac567cf6f60fcffc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:13.237103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b0c18bdc-5f49-4511-aac3-d2ba572dbad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268a7c0d30b14fd9ac567cf6f60fcffc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:13.238579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:13.260367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1085/2156] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:07:13 2022] DELETE /identity/v3/users/268a7c0d30b14fd9ac567cf6f60fcffc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:13.263363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.263902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.264552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.291288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:13.293628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.334255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b8b02-9dc9-4504-9d30-e6c0e41f17c0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.413314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1086/2157] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:13.419556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.422198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.424705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.425603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.426033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.426511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.427423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.430607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.463624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1072/2158] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:07:13 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:07:13.467932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.482404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d9576-a875-4550-a641-27fd5555470f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.490806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.492666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.493594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.494201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.495783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.497797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.514367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd4462-8507-4fde-a2ed-90ea9f2a6d37 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.520536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1073/2159] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:07:13.522622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.523025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.523692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.524319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.524715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.535232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f3088-31ee-4334-83b6-30af4b4698b8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.541031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.542145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.542870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.543371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.544326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.546177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.546590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.548058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344c4a9-3aee-43e4-b678-777958f7a828 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.565897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1074/2160] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:13.566826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.570774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1087/2161] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:07:13.585961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:13.592161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcb15c-4631-4abf-bd18-c10fdabb5c2d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:13.599878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:13.601039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.601453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.601827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] PATH_INFO: `/v3/projects/ff7ea083ba954684929cc0fdc21217eb/users/5ee6fc89c2434892bc5e725bd68db762/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.628871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7ea083ba954684929cc0fdc21217eb, user_id=5ee6fc89c2434892bc5e725bd68db762, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:13.632782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:13.649258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1075/2162] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:07:13 2022] PUT /identity/v3/projects/ff7ea083ba954684929cc0fdc21217eb/users/5ee6fc89c2434892bc5e725bd68db762/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:13.661321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:13.661599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:13.661879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:13.707847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] MFA Rules not processed for user `5ee6fc89c2434892bc5e725bd68db762`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:13.709671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.843474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1088/2163] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:13.860657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.865402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.909696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:13.911217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:13.965765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:14.029855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4966cb2-c7a1-4ac0-a8a3-88218f3338c1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:14.032994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765c617-fd16-45ef-883b-eda9127bb8bf None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:14.033828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:14.034524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:14.034810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:14.035091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:14.039358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:14.040239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:14.040520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:14.040911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] PATH_INFO: `/v3/projects/e2cf537d534d4e7fae7f415416dadb52` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:14.043213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2cf537d534d4e7fae7f415416dadb52)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:14.045113 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:14.047238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:14.049134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acc417-453d-4125-b219-00bf756b1577 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:14.066472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1076/2164] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:14.115758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2cf537d534d4e7fae7f415416dadb52', 'request_id': 'req-6a30a151-ed50-44d2-963f-9116e04174c6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:14.132805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2cf537d534d4e7fae7f415416dadb52', 'request_id': 'req-6a30a151-ed50-44d2-963f-9116e04174c6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:14.142648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2cf537d534d4e7fae7f415416dadb52', 'request_id': 'req-6a30a151-ed50-44d2-963f-9116e04174c6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:14.157406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] The token cache is being invalidate because project e2cf537d534d4e7fae7f415416dadb52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:14.157722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:14.158641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6a30a151-ed50-44d2-963f-9116e04174c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:14.160256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1089/2165] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:13 2022] DELETE /identity/v3/projects/e2cf537d534d4e7fae7f415416dadb52 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:15.175165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.175718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.175718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.177113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1077/2166] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:07:15.182801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.183278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.183362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.201969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:15.203213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:15.239642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:15.241541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:15.285415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1090/2167] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:07:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:15.297190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:15.298703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:15.311809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84926347-de5e-40a1-b505-c76f6affb9a6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:15.316921 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:15.317679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.318118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.318264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.326800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:15.328435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:15.329807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa411acc-2a68-45fd-b6f0-661b325f201f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:15.334454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:15.335275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.335557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.335828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] PATH_INFO: `/v3/users/2e84715279114df09ecfe4335c926066` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.337027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e84715279114df09ecfe4335c926066)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:15.339463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:15.341108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1091/2168] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:07:15.392313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e84715279114df09ecfe4335c926066'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.401243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e84715279114df09ecfe4335c926066'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.401895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e84715279114df09ecfe4335c926066'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.416897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e84715279114df09ecfe4335c926066'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.417745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-88ab6a79-3d33-42a3-8abe-3eb0371dc37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e84715279114df09ecfe4335c926066'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.427624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:15.432237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1078/2169] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:07:15 2022] DELETE /identity/v3/users/2e84715279114df09ecfe4335c926066 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:07:15.605740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:15.617129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34721e6-7551-455b-aa91-d79a42787244 service placement] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:15.651518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:15.678042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:15.678934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.679314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.679579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] PATH_INFO: `/v3/projects/988a6aa0bbb045bb9b66810219206ebc` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.680880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=988a6aa0bbb045bb9b66810219206ebc)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:15.683235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:15.744920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '988a6aa0bbb045bb9b66810219206ebc', 'request_id': 'req-add734e7-e675-4baa-b567-c53e32387d4e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.753738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '988a6aa0bbb045bb9b66810219206ebc', 'request_id': 'req-add734e7-e675-4baa-b567-c53e32387d4e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.760326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '988a6aa0bbb045bb9b66810219206ebc', 'request_id': 'req-add734e7-e675-4baa-b567-c53e32387d4e'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.772021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] The token cache is being invalidate because project 988a6aa0bbb045bb9b66810219206ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:15.772321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.772964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-add734e7-e675-4baa-b567-c53e32387d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:15.774621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1092/2170] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:15 2022] DELETE /identity/v3/projects/988a6aa0bbb045bb9b66810219206ebc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:15.779924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.780220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.780502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.799424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:15.800710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:15.884646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1079/2171] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:07:15.893880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.894136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.894481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.913282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:15.914596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:15.924479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1093/2172] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:15.929126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:15.939050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08b0fc-8eb1-4b1d-b7c9-9f70e295e299 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:15.943534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:15.944192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.944473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.944752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.945557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:07:15.945776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:15.947837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:15.960873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1080/2173] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:07:15.972642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:15.975683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d4ffe-6d78-420e-849e-d5a8caa77970 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:15.981185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:15.982339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:15.985741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:15.986097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:15.986922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:15.988932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.022715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1094/2174] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:07:15 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:07:16.027088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:16.037926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95eba8-1e9d-47a3-b69d-582b72eb2056 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:16.096060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:16.096712 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.096994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.097264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.097950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:16.099929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.121175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1081/2175] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:07:16 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:07:16.125774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:16.136946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391483ca-5e30-4cb7-8395-f06e3d41e35d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:16.141845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:16.142542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.142825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.143131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.143857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:16.145482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.157654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1095/2176] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:16.162195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:16.173538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f544f9-8966-48be-8d8f-8f3affe5fb14 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:16.178441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:16.179086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.179425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.179631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.180522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:16.182291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0c458-0821-4733-a5a0-e358b8327b43 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.193017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1082/2177] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:16.197428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:16.208007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2daa2-3156-4ecc-8cdb-f6509bf46182 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:16.214281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:16.215649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.216070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.216351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] PATH_INFO: `/v3/projects/792f217e834c4049b5b0d856cd53f947/users/a7ce962e4a7447848e3454905102d27c/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.234347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792f217e834c4049b5b0d856cd53f947, user_id=a7ce962e4a7447848e3454905102d27c, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:16.237105 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.253489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1096/2178] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:07:16 2022] PUT /identity/v3/projects/792f217e834c4049b5b0d856cd53f947/users/a7ce962e4a7447848e3454905102d27c/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:16.258907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971f10-a868-4a5c-8ed9-64c02969335e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.259358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971f10-a868-4a5c-8ed9-64c02969335e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.260733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be971f10-a868-4a5c-8ed9-64c02969335e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.283049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be971f10-a868-4a5c-8ed9-64c02969335e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:16.284252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be971f10-a868-4a5c-8ed9-64c02969335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:16.355637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1083/2179] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:16.365145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:16.366943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:16.422465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3e2c7-1610-4cf0-89b8-91350bc827a5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:16.426156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:16.426809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:16.427085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:16.427353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:16.436915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:16.438353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2084f1-1fb0-4c73-9f2f-b24b2b3a367a service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:16.447957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1097/2180] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:20.386627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:20.387004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:20.387182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:20.404076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:20.422799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] MFA Rules not processed for user `a7ce962e4a7447848e3454905102d27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:20.423925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:20.517495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1084/2181] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:20.528604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeef1b-c4dd-4fb3-b828-2719f4e9fe50 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:20.529141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeef1b-c4dd-4fb3-b828-2719f4e9fe50 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:20.529141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeef1b-c4dd-4fb3-b828-2719f4e9fe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:20.537283 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:20.552507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7cdeef1b-c4dd-4fb3-b828-2719f4e9fe50 None None] MFA Rules not processed for user `a7ce962e4a7447848e3454905102d27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:20.554379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7cdeef1b-c4dd-4fb3-b828-2719f4e9fe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:20.563269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1098/2182] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:20.577513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:20.580802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:20.656251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328a855-b6bb-4e16-8a1c-3e2989647c67 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:20.661715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:20.662393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:20.662673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:20.662944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:20.672367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:20.674132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:20.685327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1085/2183] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:21.351675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b972f3-bcc8-4bd5-96f3-1ed39eef4083 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:21.351968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b972f3-bcc8-4bd5-96f3-1ed39eef4083 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:21.352243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b972f3-bcc8-4bd5-96f3-1ed39eef4083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:21.371401 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-78b972f3-bcc8-4bd5-96f3-1ed39eef4083 None None] MFA Rules not processed for user `a7ce962e4a7447848e3454905102d27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:21.372876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-78b972f3-bcc8-4bd5-96f3-1ed39eef4083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:21.380606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1099/2184] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:30.680442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=63 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:30.786238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:30.787905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:30.802402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d97afc-92d1-4bac-956a-467835afef9d service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:30.807648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:30.808547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:30.808832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:30.809103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] PATH_INFO: `/v3/users/5ee6fc89c2434892bc5e725bd68db762` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:30.810431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee6fc89c2434892bc5e725bd68db762)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:30.812474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:30.862552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee6fc89c2434892bc5e725bd68db762'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:30.871116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee6fc89c2434892bc5e725bd68db762'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:30.871814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee6fc89c2434892bc5e725bd68db762'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:30.885819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee6fc89c2434892bc5e725bd68db762'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:30.886741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee6fc89c2434892bc5e725bd68db762'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:30.901423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1086/2185] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:07:30 2022] DELETE /identity/v3/users/5ee6fc89c2434892bc5e725bd68db762 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:30.906280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:30.906814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:30.906974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:30.924767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:30.925980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:30.990975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1100/2186] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:07:31.001187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:31.002770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:31.064283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed6488-0f2d-4e87-9db0-2c94b5ff62b7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:31.068385 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:31.069044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:31.069326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:31.069596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:31.080098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:31.081571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:31.091697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1087/2187] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:31.368428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:31.370126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:31.396654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8287535-f403-4919-9b12-697a5db13c0e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:31.402353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:31.403263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:31.403543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:31.403817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] PATH_INFO: `/v3/projects/ff7ea083ba954684929cc0fdc21217eb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:31.405092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7ea083ba954684929cc0fdc21217eb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:31.407646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:31.472491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7ea083ba954684929cc0fdc21217eb', 'request_id': 'req-3f3fc186-ce4e-49c9-ba09-77c816e41012'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.487526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7ea083ba954684929cc0fdc21217eb', 'request_id': 'req-3f3fc186-ce4e-49c9-ba09-77c816e41012'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.499021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7ea083ba954684929cc0fdc21217eb', 'request_id': 'req-3f3fc186-ce4e-49c9-ba09-77c816e41012'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.511190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:31.512588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:31.515683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] The token cache is being invalidate because project ff7ea083ba954684929cc0fdc21217eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:31.516180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.516974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-3f3fc186-ce4e-49c9-ba09-77c816e41012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.519032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1101/2188] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:31 2022] DELETE /identity/v3/projects/ff7ea083ba954684929cc0fdc21217eb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:31.525912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853e883-7032-4d2e-ab67-81d78fa49a44 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:31.593704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:31.594790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:31.595123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:31.595475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] PATH_INFO: `/v3/users/4b279e5cb18e484d8e4997d5d35b9935` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:31.596931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b279e5cb18e484d8e4997d5d35b9935)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:31.599083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:31.619351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:31.667713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b279e5cb18e484d8e4997d5d35b9935'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.685645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b279e5cb18e484d8e4997d5d35b9935'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.688077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b279e5cb18e484d8e4997d5d35b9935'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.712840 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b279e5cb18e484d8e4997d5d35b9935'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.713935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b279e5cb18e484d8e4997d5d35b9935'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:31.738253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1088/2189] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:07:31 2022] DELETE /identity/v3/users/4b279e5cb18e484d8e4997d5d35b9935 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:07:31.743786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:31.744192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:31.744626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:31.765783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:31.767967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:31.831613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1102/2190] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:31.841178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:31.843051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:31.901500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130572be-e1b6-4d33-8e44-1d92d2583b76 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:31.905238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:31.905885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:31.906161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:31.906465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:31.916248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:31.917736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5f7b3-7ead-4ea2-90ea-5777af7f455e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:31.928595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1089/2191] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:07:32.230606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.232933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:32.248738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950a502-7acd-4246-92ca-86a3f7daba88 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.254202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.255351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.255725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.256125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] PATH_INFO: `/v3/projects/b19521d94fcc490ebc947a9cbdd733ea` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.259215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19521d94fcc490ebc947a9cbdd733ea)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.261314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.277121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:32.316760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19521d94fcc490ebc947a9cbdd733ea', 'request_id': 'req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:32.324807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19521d94fcc490ebc947a9cbdd733ea', 'request_id': 'req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:32.332202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19521d94fcc490ebc947a9cbdd733ea', 'request_id': 'req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:32.343160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] The token cache is being invalidate because project b19521d94fcc490ebc947a9cbdd733ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:07:32.343447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:32.344089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b7bf9c6a-df94-43e6-9b64-c1c41d3cffdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:07:32.345657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1103/2192] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:07:32 2022] DELETE /identity/v3/projects/b19521d94fcc490ebc947a9cbdd733ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:32.350603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.350895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.351197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.371121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:32.372440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:32.433329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1090/2193] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:32.441474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.441807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.442128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.461003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:32.462205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:32.470589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1104/2194] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:32.475449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.489295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0e8c9-18e0-43dc-9bde-ca02ede630e7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.497376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.498366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.498639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.498899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.499794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:07:32.500182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.502728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.520550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1091/2195] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:07:32.521943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.531694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdefcc9-1fc0-4f13-bd27-1b8028ad904c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.544075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.545099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.545529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.546481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.546948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.549632 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.586287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1105/2196] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:07:32 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:07:32.587733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.602301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bf0ce-646d-45c0-a969-31256b349fa1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.629218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:32.670172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.671175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.671550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.671905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.672619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.674596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.698924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1092/2197] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:07:32 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:07:32.701535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.712062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ea18b-e704-4907-b5d2-35e63d845efc admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.717250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.718121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.718500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.718790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.719513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.721134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.731978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1106/2198] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:07:32.737160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.751257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beabcc-5d24-4f79-977b-4662c64f303d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.756423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.757090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.757361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.757631 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.758408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.760023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab1f8b-7388-4967-9c5d-a3cb790619f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.770583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1093/2199] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:07:32.774881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.785391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ec218-fa82-469a-bb71-ba357d0c2936 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:32.790349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:32.791258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.791541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.791821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] PATH_INFO: `/v3/projects/5749eb9aa81d4b5ea7603961fd977c76/users/1c945760a6cf492890847e18bfc9e16a/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.808455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5749eb9aa81d4b5ea7603961fd977c76, user_id=1c945760a6cf492890847e18bfc9e16a, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:32.811277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:32.827421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1107/2200] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:07:32 2022] PUT /identity/v3/projects/5749eb9aa81d4b5ea7603961fd977c76/users/1c945760a6cf492890847e18bfc9e16a/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:07:32.838798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:32.839013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:32.839286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:32.867645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] MFA Rules not processed for user `1c945760a6cf492890847e18bfc9e16a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:07:32.868915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:32.938344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1094/2201] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:07:32.950551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:07:32.952459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:07:33.014756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6a7fa-7eed-45d1-b651-94495abd4ec1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:07:33.019039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:07:33.019680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:07:33.019961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:07:33.020233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:07:33.029888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:07:33.031375 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31178798-42e6-4298-805a-71c182d2e2ac service glance] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:07:33.040864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1108/2202] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:07:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:07:42.934995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:07:43.036181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:18.407872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:18.409466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:18.482346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b82216-476d-4204-bdef-898d86f3e67a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:18.487314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:18.488242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:18.488520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:18.488786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] PATH_INFO: `/v3/users/6cb44f0155eb423a9330a71410f3461d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:18.490018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb44f0155eb423a9330a71410f3461d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:18.492033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:18.541936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb44f0155eb423a9330a71410f3461d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:18.552542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb44f0155eb423a9330a71410f3461d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:18.553432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb44f0155eb423a9330a71410f3461d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:18.567388 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb44f0155eb423a9330a71410f3461d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:18.568095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb44f0155eb423a9330a71410f3461d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:18.581673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1095/2203] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:08:18 2022] DELETE /identity/v3/users/6cb44f0155eb423a9330a71410f3461d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:18.586457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:18.586936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:18.587041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:18.605647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:18.606837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:18.675805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1109/2204] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:18.685973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:18.687806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:18.756258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9436766-439e-4727-b8fd-987365191195 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:18.763110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:18.763773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:18.764053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:18.764349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:18.778517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:18.779976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38552c8b-623d-4772-b012-f8c28299e97c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:18.790087 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1096/2205] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:08:19.092233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.093763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:19.106036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e907e93-f211-4707-83db-a54dc8b7a73d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.110758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.111663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.111939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.112210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] PATH_INFO: `/v3/projects/7246586d5ea74352a1676308edab1fe8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.113436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7246586d5ea74352a1676308edab1fe8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.115444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.165611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7246586d5ea74352a1676308edab1fe8', 'request_id': 'req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:19.174661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7246586d5ea74352a1676308edab1fe8', 'request_id': 'req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:19.180992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7246586d5ea74352a1676308edab1fe8', 'request_id': 'req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:19.192291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] The token cache is being invalidate because project 7246586d5ea74352a1676308edab1fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:08:19.192576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:19.193221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0f5c8ce8-bab2-4a5c-8367-ec0e70d83844 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:19.194803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1110/2206] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:08:19 2022] DELETE /identity/v3/projects/7246586d5ea74352a1676308edab1fe8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:19.200334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.200659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.200969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.220550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:19.221841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:19.243299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:19.289845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1097/2207] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:19.298044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.298342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.298634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.318002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:19.319241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:19.328923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1111/2208] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:19.333466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.343874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544b7ac-5a74-4075-ac77-38cd31964f58 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.348619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.349279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.349555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.349825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.350949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:08:19.351260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.352888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.365244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1098/2209] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:08:19.369711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.379505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b45a13-8821-4c6c-b0e9-a7741890a704 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.384739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.385397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.385675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.385982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.386712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.388622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.417164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1112/2210] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:08:19 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:19.421361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.431934 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac4080-0f03-42b3-90e7-d6354dd5af8c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.489417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.490105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.490527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.490807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.491498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.493429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.516950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1099/2211] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:08:19 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:19.521418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.532962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79865f-7b11-4b8f-a4b1-87f8bf74a0f0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.538211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.538900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.539181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.539449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.540185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.541838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.552903 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1113/2212] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:08:19.557004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.567064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b57222-30b4-4192-a199-11fd86297e18 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.571729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.572370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.572650 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.572923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.573646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.575417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ece36-b4f6-4e3d-a952-a2a24d4b4c78 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.585519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1100/2213] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:08:19.589690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.600374 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835f8ac-ff3a-4911-9d0e-5f70cb48a1ef admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.606077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.606969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.607246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.607516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] PATH_INFO: `/v3/projects/22b3073c2c6f4f479aae8d37cccc7ae8/users/79388c8109554a82b6336fd8c803005f/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.622940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b3073c2c6f4f479aae8d37cccc7ae8, user_id=79388c8109554a82b6336fd8c803005f, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.625588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.642215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1114/2214] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:08:19 2022] PUT /identity/v3/projects/22b3073c2c6f4f479aae8d37cccc7ae8/users/79388c8109554a82b6336fd8c803005f/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:19.647523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77fd6-886b-477c-a0ed-5ba67f607d79 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.647819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77fd6-886b-477c-a0ed-5ba67f607d79 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.648104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e77fd6-886b-477c-a0ed-5ba67f607d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.666165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4e77fd6-886b-477c-a0ed-5ba67f607d79 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:19.667397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4e77fd6-886b-477c-a0ed-5ba67f607d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:19.735181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1101/2215] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:19.745610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:19.748307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:19.775304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:19.814488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429a738-b762-4301-8886-b3765be36f7e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:19.818640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:19.819284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:19.819570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:19.820243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:19.829480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:19.830939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:19.841186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1115/2216] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:20.513820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d14ca-40cd-4498-9f5a-c3db973f9120 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:20.514489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d14ca-40cd-4498-9f5a-c3db973f9120 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:20.514489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d14ca-40cd-4498-9f5a-c3db973f9120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:20.564573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ef5d14ca-40cd-4498-9f5a-c3db973f9120 None None] MFA Rules not processed for user `79388c8109554a82b6336fd8c803005f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:20.565985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ef5d14ca-40cd-4498-9f5a-c3db973f9120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:20.599291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:20.758267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1102/2217] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:20.766270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:20.767997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:20.852953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c723-5f5d-4889-a56e-0a96ee1c7bf3 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:20.856499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:20.857301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:20.857541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:20.857804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:20.867030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:20.868476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:20.878107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1116/2218] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:22.240967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d81f-34da-4fad-85df-aeb1a57b5220 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:22.241562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d81f-34da-4fad-85df-aeb1a57b5220 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:22.241562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6d81f-34da-4fad-85df-aeb1a57b5220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:22.261690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-67a6d81f-34da-4fad-85df-aeb1a57b5220 None None] MFA Rules not processed for user `79388c8109554a82b6336fd8c803005f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:22.263010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-67a6d81f-34da-4fad-85df-aeb1a57b5220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:22.270687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1103/2219] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:30.873756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=52 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:32.255558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=34 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:33.069306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:33.071474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:33.130213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63015d9d-22a4-42d2-a844-ac462bcd21d5 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:33.135126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:33.136012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:33.136285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:33.136554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] PATH_INFO: `/v3/users/1c945760a6cf492890847e18bfc9e16a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:33.137829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c945760a6cf492890847e18bfc9e16a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:33.139787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:33.189938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c945760a6cf492890847e18bfc9e16a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.199281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c945760a6cf492890847e18bfc9e16a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.199919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c945760a6cf492890847e18bfc9e16a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.213027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c945760a6cf492890847e18bfc9e16a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.213914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c945760a6cf492890847e18bfc9e16a'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.229389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1117/2220] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:08:33 2022] DELETE /identity/v3/users/1c945760a6cf492890847e18bfc9e16a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:33.234261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:33.234575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:33.234850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:33.264119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:33.264834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:33.325957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1104/2221] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:08:33.336189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:33.337839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:33.432761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed83-248b-414c-9353-981b469ca0bb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:33.437615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:33.438289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:33.438607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:33.438881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:33.449606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:33.451082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a36e65-302b-4326-9af7-334cb5efd031 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:33.461138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1118/2222] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:33.753435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:33.755069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:33.767971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f4d8d-d2f9-4d52-8172-a99b8266a10d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:33.773282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:33.774184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:33.774491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:33.774771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] PATH_INFO: `/v3/projects/5749eb9aa81d4b5ea7603961fd977c76` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:33.776038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5749eb9aa81d4b5ea7603961fd977c76)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:33.778013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:33.826838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5749eb9aa81d4b5ea7603961fd977c76', 'request_id': 'req-c7bcb640-b7b4-496c-a87c-e31994d88891'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.835169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5749eb9aa81d4b5ea7603961fd977c76', 'request_id': 'req-c7bcb640-b7b4-496c-a87c-e31994d88891'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.841825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5749eb9aa81d4b5ea7603961fd977c76', 'request_id': 'req-c7bcb640-b7b4-496c-a87c-e31994d88891'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.859761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] The token cache is being invalidate because project 5749eb9aa81d4b5ea7603961fd977c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:08:33.860050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.860774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7bcb640-b7b4-496c-a87c-e31994d88891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:33.862395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1105/2223] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:08:33 2022] DELETE /identity/v3/projects/5749eb9aa81d4b5ea7603961fd977c76 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:33.867799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:33.867965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:33.868272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:33.898513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:33.924412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:33.925636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:33.994603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1119/2224] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:34.002570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.002858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.003135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.023917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:34.025133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:34.035465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1106/2225] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:34.039890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.050149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db34c75-ea43-4c68-989f-41a9f0be3491 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.055042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.055712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.056016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.056309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.057040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:08:34.057343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.059046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.071042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1120/2226] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:08:34.075625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.085297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5c3b3-d33e-4c9a-80a2-419809b5392d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.091171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.091834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.092112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.092381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.093078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.095042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.125243 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1107/2227] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:08:34 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:34.126445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.136996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a500ed6-4a04-4630-af1c-6c8da84a4eab admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.195035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.195733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.196015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.196297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.197001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.198978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.222817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1121/2228] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:08:34 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:34.224072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.234664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89112c-8255-4421-9029-5f34248996d2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.239967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.240624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.241023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.241287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.242165 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.243817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.254888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1108/2229] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:08:34.258859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.269356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772612e-19dc-40b3-8de5-fcf2af5d071b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.274023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.274706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.274986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.275256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.275984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.277590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7bdb2-a245-4576-8a73-c84237012ca3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.287652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1122/2230] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:08:34.291911 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.302665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001919e-ce6d-459b-8ec7-a1c9cf540832 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.307752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.308633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.308927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.309200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] PATH_INFO: `/v3/projects/7bbaac405c664c68839edc4ae9f47c7d/users/aaee445be2024855883ae51573355c17/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.324774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bbaac405c664c68839edc4ae9f47c7d, user_id=aaee445be2024855883ae51573355c17, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.327463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.343841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1109/2231] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:08:34 2022] PUT /identity/v3/projects/7bbaac405c664c68839edc4ae9f47c7d/users/aaee445be2024855883ae51573355c17/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:34.356733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.357023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.357305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.386742 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] MFA Rules not processed for user `aaee445be2024855883ae51573355c17`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:34.387976 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:34.438485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:34.462545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1123/2232] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:34.474154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.476054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:34.490439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:34.537306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830f72-df12-4c3e-b74e-aa8463933a42 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.541224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.541876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.542154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.542459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.551022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.552644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0315a2-39ec-4eed-90f9-c5e31a739eb9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.562050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1110/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:08:34.586900 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.653221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62b3d1-8991-4923-991b-60532400cd08 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.658133 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.659146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.659427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.659702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] PATH_INFO: `/v3/users/aaee445be2024855883ae51573355c17` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.660932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaee445be2024855883ae51573355c17)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.663019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.713831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaee445be2024855883ae51573355c17'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:34.724146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaee445be2024855883ae51573355c17'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:34.724780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaee445be2024855883ae51573355c17'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:34.738284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaee445be2024855883ae51573355c17'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:34.738978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaee445be2024855883ae51573355c17'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:34.751541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1124/2234] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:08:34 2022] DELETE /identity/v3/users/aaee445be2024855883ae51573355c17 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:34.756273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.756637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.756909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.777609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:34.778865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:34.839619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1111/2235] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:34.849914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:34.851736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:34.914933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb74002-9ea0-4c92-a653-0980123a8488 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:34.919221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:34.919870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:34.920148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:34.920416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:34.930614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:34.932047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff79b5d-e1fb-4763-b745-1efb92fa11af service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:34.941910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1125/2236] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:35.224488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.225957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:35.238777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13148e6-21d0-40a0-a036-76e882a13e7c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.244381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.245311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.245553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.245828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] PATH_INFO: `/v3/projects/7bbaac405c664c68839edc4ae9f47c7d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.247139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bbaac405c664c68839edc4ae9f47c7d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.249091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.326423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bbaac405c664c68839edc4ae9f47c7d', 'request_id': 'req-7cc04def-976f-4279-a909-ba42269bb652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:35.335838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bbaac405c664c68839edc4ae9f47c7d', 'request_id': 'req-7cc04def-976f-4279-a909-ba42269bb652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:35.342939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bbaac405c664c68839edc4ae9f47c7d', 'request_id': 'req-7cc04def-976f-4279-a909-ba42269bb652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:35.354137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] The token cache is being invalidate because project 7bbaac405c664c68839edc4ae9f47c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:08:35.354454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:35.355110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cc04def-976f-4279-a909-ba42269bb652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:08:35.356931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1112/2237] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:08:35 2022] DELETE /identity/v3/projects/7bbaac405c664c68839edc4ae9f47c7d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:35.362944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.363491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.363491 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.381859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:35.383187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:35.391978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:35.444863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1126/2238] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:35.452931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.453178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.453459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.473121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:35.474298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:35.482579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1113/2239] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:35.487288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.498139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7aba3-48c4-4ccc-ae4b-b695da396896 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.503507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.504176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.504454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.504728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.505486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:08:35.505792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.507432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.520176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1127/2240] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:08:35.524774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.534561 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0dfed4-b259-47e3-a1e4-1951c8ac6113 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.540181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.541102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.541382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.541826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.543178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.544542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.571778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1114/2241] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:08:35 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:35.575989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.586270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f34326-76d7-4c6f-b790-0ad3fa19b676 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.649354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.650026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.650308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.650609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.651311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.653240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.675456 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1128/2242] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:08:35 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:35.679652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.692190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef3d9c-1835-4777-a1fd-f4d127128008 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.699239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.699924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.700202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.700478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.701218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.702900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.708999 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:35.714869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1115/2243] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:08:35.719560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.730183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b6952-8c72-480f-9075-796fbe7c49e1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.735080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.735754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.736038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.736314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.737040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.738688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b176b9-be42-4f92-984b-d7a92bf581ca admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.748803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1129/2244] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:08:35.753167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.766859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df562f-e49f-4d0f-93b6-8fd1385bdde8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.772922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.773813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.774092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.774343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] PATH_INFO: `/v3/projects/19004d450eee426786ca037d00f8fff4/users/9e51be11f09d44588e5b4ef657907fe6/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.789856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19004d450eee426786ca037d00f8fff4, user_id=9e51be11f09d44588e5b4ef657907fe6, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.792572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:35.807692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1116/2245] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:08:35 2022] PUT /identity/v3/projects/19004d450eee426786ca037d00f8fff4/users/9e51be11f09d44588e5b4ef657907fe6/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:08:35.812979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84198-5516-4598-b753-8a0df56419f5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.813274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84198-5516-4598-b753-8a0df56419f5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.813546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd84198-5516-4598-b753-8a0df56419f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.832252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5cd84198-5516-4598-b753-8a0df56419f5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:35.833470 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5cd84198-5516-4598-b753-8a0df56419f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:35.903000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1130/2246] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:35.912675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:35.915417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:35.976584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc351d3-754a-4572-a2ea-11fc71d6ed00 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:35.981149 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:35.981788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:35.982060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:35.982344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:35.991995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:35.993904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:36.003658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1117/2247] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:40.361797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.362277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.362671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.394747 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] MFA Rules not processed for user `9e51be11f09d44588e5b4ef657907fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:40.396383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:40.405982 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:40.483024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1131/2248] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:40.487320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.498166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740769f-c6b8-4b6f-b138-d392eba09fdf service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.508545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.510181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.510730 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.511160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.512131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.514309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.555716 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1118/2249] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:08:40 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:40.558574 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.570212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7965ac-17a7-4048-8a7d-2bf036ce6006 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.635223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.635954 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.636241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.636522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.637234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.639281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.662708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1132/2250] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:08:40 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:08:40.667297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.679676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310228b5-5bcc-4fad-84ad-32ad10591944 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.684815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.685580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.685930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.686261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.687129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.688890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.700435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1119/2251] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:08:40.704655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.715917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a688d57-343e-4544-99df-3870feeae9c0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.720555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.721452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.721738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.721998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] PATH_INFO: `/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5/users/9c9890d6755f4d77bfd2cb77694da9fb/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.759021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e17d7c66ac46629e5e0c5f2ad659e5, user_id=9c9890d6755f4d77bfd2cb77694da9fb, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.762647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.783654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1133/2252] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:08:40 2022] PUT /identity/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5/users/9c9890d6755f4d77bfd2cb77694da9fb/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:40.787797 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.798454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0eb0d-31f3-4237-872f-7690793bdd47 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.870043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.870841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.871126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.871438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.872448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.874190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12cc53-a636-4df1-98dd-a0231e25f5d6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.885746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1120/2253] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:08:40.889738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:40.900724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbac3b-af50-412a-8914-21e4eda80729 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:40.905414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:40.906338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:40.906627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:40.906897 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] PATH_INFO: `/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5/users/9c9890d6755f4d77bfd2cb77694da9fb/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:40.908949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35e17d7c66ac46629e5e0c5f2ad659e5, user_id=9c9890d6755f4d77bfd2cb77694da9fb, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:40.911651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:40.922911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1134/2254] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:08:40 2022] PUT /identity/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5/users/9c9890d6755f4d77bfd2cb77694da9fb/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:08:45.900773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077e4e-32c2-4f24-ba2e-1d82d4bd6262 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:45.901250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077e4e-32c2-4f24-ba2e-1d82d4bd6262 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:45.901342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2077e4e-32c2-4f24-ba2e-1d82d4bd6262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:45.931445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e2077e4e-32c2-4f24-ba2e-1d82d4bd6262 None None] MFA Rules not processed for user `9c9890d6755f4d77bfd2cb77694da9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:45.932618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e2077e4e-32c2-4f24-ba2e-1d82d4bd6262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:45.967231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:46.016357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1121/2255] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:45 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:46.034152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:46.039737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:46.135344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:08:46.175194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11733c3b-c749-4345-afa0-e165e71b1e18 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:46.181891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:46.182567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:46.182862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:46.183131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:46.250463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:46.252135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0def4a-9932-4dda-bec1-6a4948e86839 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:46.267074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1135/2256] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:08:47.448435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:47.448985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:47.448985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:47.469192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] MFA Rules not processed for user `9e51be11f09d44588e5b4ef657907fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:47.470476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:47.479073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1122/2257] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:52.582681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474659a-b39c-4e41-831d-01bf841465a3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:52.583187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474659a-b39c-4e41-831d-01bf841465a3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:52.583319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474659a-b39c-4e41-831d-01bf841465a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:52.605822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6474659a-b39c-4e41-831d-01bf841465a3 None None] MFA Rules not processed for user `a7ce962e4a7447848e3454905102d27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:08:52.607111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6474659a-b39c-4e41-831d-01bf841465a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:52.665503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1136/2258] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:08:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:08:55.233238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:08:55.234794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:55.328257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68740102-6888-46ad-b7ea-b2c244459d58 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:08:55.333784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:08:55.334492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:08:55.334770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:08:55.335045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:08:55.336961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:08:55.349604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:08:55.351260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81f57-8eac-4b2d-b4e1-8a6df6b14bd7 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:08:55.362092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1123/2259] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:08:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:09:02.657353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=65 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:09:05.357225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=59 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:09:20.931860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:09:20.932391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:09:20.932391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:09:20.951427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] MFA Rules not processed for user `126e2b6fc7114ec3bdc1bb3a99fcf752`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:09:20.952638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:09:21.015495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1137/2260] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:09:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:09:31.003307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=25 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:09:48.689714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520629b-e153-4044-b677-84543886c5ef None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:09:48.690285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520629b-e153-4044-b677-84543886c5ef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:09:48.690285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520629b-e153-4044-b677-84543886c5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:09:48.753306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d520629b-e153-4044-b677-84543886c5ef None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:09:48.754581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d520629b-e153-4044-b677-84543886c5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:09:48.825801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1124/2261] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:09:48.878739 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:09:48.881103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:09:49.021453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35f35c-0bf9-4ead-afa3-1629d72e97bc None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:09:49.027118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:09:49.028587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:09:49.028945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:09:49.029223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:09:49.039859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:09:49.042442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1da7d-899f-4036-bd7c-e2ca79e163d0 service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:09:49.051302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1138/2262] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:09:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:09:58.826197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=44 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:09:59.047893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=41 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:10:09.018629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.020037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.020565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.022144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1125/2263] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:10:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:10:09.028904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f624-3c32-4d9b-8092-31837ea46bc0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.029428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f624-3c32-4d9b-8092-31837ea46bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.029559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790f624-3c32-4d9b-8092-31837ea46bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.072701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-2790f624-3c32-4d9b-8092-31837ea46bc0 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:09.074082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2790f624-3c32-4d9b-8092-31837ea46bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:09.084840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1139/2264] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:09.103452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:09.104940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:09.189910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be313a54-a3e4-4d12-b4fe-6c33c992438a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:09.193503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:09.194142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.194456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.194757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.204742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:09.206190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778dec1-c70c-4b55-938c-7ab2e83f37dd service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:09.217085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1126/2265] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:10:09.302462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d0b1-236d-4395-83d5-7479bf0e5129 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.302760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d0b1-236d-4395-83d5-7479bf0e5129 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.303033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3d0b1-236d-4395-83d5-7479bf0e5129 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.304526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1140/2266] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:10:09.310386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ddc0-59f2-47a7-a7c9-6075d7a99e27 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.310749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ddc0-59f2-47a7-a7c9-6075d7a99e27 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.311135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c0ddc0-59f2-47a7-a7c9-6075d7a99e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.339512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d7c0ddc0-59f2-47a7-a7c9-6075d7a99e27 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:09.340742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d7c0ddc0-59f2-47a7-a7c9-6075d7a99e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:09.400331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1127/2267] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:09.414505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3d0b1-236d-4395-83d5-7479bf0e5129 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:09.427198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3d0b1-236d-4395-83d5-7479bf0e5129 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:09.431507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:09.432166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.432445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.432717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.440786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:09.442269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f7e3c-9ef9-4028-8e92-ec6006f50125 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:09.450722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1141/2268] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:10:09.542637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.542908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.543280 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.545377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1128/2269] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:10:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:10:09.551318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7975-5751-4bf2-bee9-58c5c566ebe3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.551607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7975-5751-4bf2-bee9-58c5c566ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.551918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7975-5751-4bf2-bee9-58c5c566ebe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.574599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d57b7975-5751-4bf2-bee9-58c5c566ebe3 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:09.575814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d57b7975-5751-4bf2-bee9-58c5c566ebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:09.587353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1142/2270] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:09.600371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:09.602582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:09.617978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29107e5-cb50-4511-a38d-38d1a51363df None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:09.622921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:09.623554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:09.623836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:09.624126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:09.635393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:09.636828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:09.647487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1129/2271] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:10:14.182749 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19a87-49bb-4061-8c3b-17b011faf3bb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:14.183246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19a87-49bb-4061-8c3b-17b011faf3bb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:14.183246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb19a87-49bb-4061-8c3b-17b011faf3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:14.202027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-bcb19a87-49bb-4061-8c3b-17b011faf3bb None None] MFA Rules not processed for user `79388c8109554a82b6336fd8c803005f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:14.203278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bcb19a87-49bb-4061-8c3b-17b011faf3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:14.262716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1143/2272] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:15.536747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:15.545925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80460bd-05cd-44a5-a9c2-57414ea6df7f service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:15.549829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:15.550504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:15.550786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:15.551056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:15.560220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:15.561732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:15.571950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1130/2273] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:10:18.659089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:18.659593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:18.659788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:18.684637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:18.687057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:18.696354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1144/2274] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:10:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:18.709292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:18.722875 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52306e-9257-4977-9e20-8b2712a24f3b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:18.730740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:18.731692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:18.732558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:18.733018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:18.746936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:18.748541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:18.758076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1131/2275] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:10:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:10:28.676240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=46 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:10:28.754450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=76 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:10:42.738586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:42.749904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c29031-0e80-4c4b-859a-a292351c57fb None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:42.753637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:42.754269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:42.754587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:42.754861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:42.756660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:42.770403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:42.771835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c71ac-ef11-40c0-be32-1809855a5d27 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:42.782817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1145/2276] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:10:43.231856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:43.242252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc25d76-5c2c-4f35-ab82-b6de1f79bebe service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:43.247950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:43.248884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:43.249256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:43.249665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:43.252989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:43.270137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:43.271643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:43.282925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1132/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:10:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:10:52.778203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:10:53.277855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:10:55.124705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:55.125203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:55.125300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:55.143821 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:10:55.145025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:10:55.153249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1146/2278] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:10:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:10:55.176185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:10:55.188910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197b347-e1c0-4742-acaa-0f190c434b18 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:10:55.195341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:10:55.195993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:10:55.196282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:10:55.196545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:10:55.205310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:10:55.206946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67237905-84ef-4181-992e-e3bb93997e83 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:10:55.215592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1133/2279] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:10:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:11:05.138482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:11:05.212118 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:11:23.213792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:11:23.215700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:23.232807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa4a96-73f7-412e-84bc-eaf1c0cfa11d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:11:23.238416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:11:23.239065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:23.239350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:23.239628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:23.241393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:23.255511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:11:23.257123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc7dcf-9403-4266-9a67-1bb0c056a19e service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:11:23.268803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1147/2280] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:11:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:11:33.263916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:11:51.760806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990031b-fd79-45ed-8805-e5402f515b18 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:51.761377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990031b-fd79-45ed-8805-e5402f515b18 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:51.761377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990031b-fd79-45ed-8805-e5402f515b18 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:51.762787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1134/2281] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:11:51.773874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:51.774436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:51.774552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:51.811951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] MFA Rules not processed for user `998f1e616d984e7fb7feac65a79c5599`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:11:51.813181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:51.822792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1148/2282] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:11:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:11:51.835318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990031b-fd79-45ed-8805-e5402f515b18 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:11:51.836256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:11:51.838281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:51.851576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990031b-fd79-45ed-8805-e5402f515b18 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:11:51.857008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:11:51.857705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:51.857985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:51.858257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:51.867734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:11:51.869255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:11:51.879457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1135/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:11:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:11:51.909524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b92af-a4a8-487c-8f68-bcd038cfa9b5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:11:51.914477 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:11:51.915382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:51.915663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:51.915936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] PATH_INFO: `/v3/users/79388c8109554a82b6336fd8c803005f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:51.917144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79388c8109554a82b6336fd8c803005f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:11:51.919114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:11:51.977269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79388c8109554a82b6336fd8c803005f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:51.988718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79388c8109554a82b6336fd8c803005f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:51.989464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79388c8109554a82b6336fd8c803005f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.002393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79388c8109554a82b6336fd8c803005f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.003103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79388c8109554a82b6336fd8c803005f'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.008202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:11:52.016543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1149/2284] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:11:51 2022] DELETE /identity/v3/users/79388c8109554a82b6336fd8c803005f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:11:52.017992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db38345-0d26-4e8a-a34c-fdc629eab6f0 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:11:52.079074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:11:52.079889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:52.080313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:52.081287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:52.082803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:52.093427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:11:52.095000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:11:52.103543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1136/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:11:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:11:52.275583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:11:52.277246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:11:52.378730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a68e13-bfe6-4805-b5db-69ff61cb1554 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:11:52.384387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:11:52.385263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:11:52.385541 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:11:52.385813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] PATH_INFO: `/v3/projects/22b3073c2c6f4f479aae8d37cccc7ae8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:11:52.387091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b3073c2c6f4f479aae8d37cccc7ae8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:11:52.389044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:11:52.391444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:11:52.470245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b3073c2c6f4f479aae8d37cccc7ae8', 'request_id': 'req-36594483-59bb-4cfe-b1d3-7c0d3944190b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.482444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b3073c2c6f4f479aae8d37cccc7ae8', 'request_id': 'req-36594483-59bb-4cfe-b1d3-7c0d3944190b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.490804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b3073c2c6f4f479aae8d37cccc7ae8', 'request_id': 'req-36594483-59bb-4cfe-b1d3-7c0d3944190b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.507438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] The token cache is being invalidate because project 22b3073c2c6f4f479aae8d37cccc7ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:11:52.507914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.508816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-36594483-59bb-4cfe-b1d3-7c0d3944190b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:11:52.510920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1150/2286] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:11:52 2022] DELETE /identity/v3/projects/22b3073c2c6f4f479aae8d37cccc7ae8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:12:02.099947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=32 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:02.361811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:02.363495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:02.438872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64da51d-9a29-4fc4-bb32-83409926366c service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:02.505069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:02.505069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:02.505069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:02.505697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:02.506460 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:02.507122 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:02.573132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:02.574778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:02.583941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1137/2287] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:12.580004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:14.308168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:14.308691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:14.308691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:14.329047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] MFA Rules not processed for user `a7ce962e4a7447848e3454905102d27c`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:12:14.330404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:14.392252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1151/2288] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:12:20.749171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:20.750748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:20.771273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7111e-27e7-42e1-a82a-4600dcd6e046 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:20.779828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:20.780942 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:20.781448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:20.781914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:20.785048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:20.849281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:20.850939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:20.860904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1138/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:24.384425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=25 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:24.560140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:24.576427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e03f12-f433-4443-80d3-9e11a7ab3dd7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:24.583156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:24.583950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:24.584331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:24.585666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:24.586786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:24.601638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:24.603344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf5dab-1548-4354-a303-5c9f12371679 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:24.615825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1152/2290] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:29.559471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:29.562175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:29.582682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dc5e7-f60d-485a-b3de-17d18c4997e1 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:29.589323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:29.589963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:29.590245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:29.590559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:29.592438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:29.658878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:29.661192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:29.673450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1139/2291] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:34.610114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:34.673343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:34.673630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:34.674115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:34.694643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] MFA Rules not processed for user `126e2b6fc7114ec3bdc1bb3a99fcf752`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:12:34.695884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:34.704589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1153/2292] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:12:35.231155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:35.233474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:35.302425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032a26-3492-4670-943b-18c8c0b8cc58 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:35.306355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:35.307022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:35.307305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:35.307580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:35.309489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:35.324476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:35.325937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1edf24-1caf-4af2-88a6-599e72136de5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:35.334429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1140/2293] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:35.817272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:35.819508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:35.924205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654e8bb-c1a9-4394-bce0-5922480c420d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:35.939388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:35.940969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:35.941420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:35.941860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:35.946140 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:35.977726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:35.979501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:36.010710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1154/2294] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:12:40.362064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f176-2251-4935-b63d-5edc871abaee None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:40.362594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f176-2251-4935-b63d-5edc871abaee None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:40.362695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971f176-2251-4935-b63d-5edc871abaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:40.381349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e971f176-2251-4935-b63d-5edc871abaee None None] MFA Rules not processed for user `e5e99d5484594d13b113f27e77b37efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:12:40.382663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e971f176-2251-4935-b63d-5edc871abaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:40.390693 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1141/2295] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:12:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:12:45.992123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=35 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:46.959040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:12:46.978131 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54eb4bf-b4af-44a0-95f4-7d5f55778899 service placement] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:12:46.987402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:12:46.988617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:12:46.989293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:12:46.989831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:12:46.992682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:12:47.006892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:12:47.008576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff94cbc-a006-483e-b303-1c75c70c490e service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:12:47.021107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1155/2296] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:12:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:12:50.375713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:12:57.016055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:26.039394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971f176-2251-4935-b63d-5edc871abaee None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:26.040841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e971f176-2251-4935-b63d-5edc871abaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:26.123391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e971f176-2251-4935-b63d-5edc871abaee None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:26.127536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:26.128181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:26.128495 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:26.132600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:26.135053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:26.148651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:26.150233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2730f-eadd-4518-a74f-32a7922c5cad service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:26.159812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1142/2297] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:30.226915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888482ab-0023-4f98-b692-529f0bd38578 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:30.227427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888482ab-0023-4f98-b692-529f0bd38578 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:30.227427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888482ab-0023-4f98-b692-529f0bd38578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:30.245011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-888482ab-0023-4f98-b692-529f0bd38578 None None] MFA Rules not processed for user `e0df4b789a804a00a267bcc5cf778324`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:30.246269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-888482ab-0023-4f98-b692-529f0bd38578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:30.253267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1156/2298] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:36.155844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=35 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:40.240230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:41.992266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714464-97c4-41c5-82fe-90d753135b19 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:41.992872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714464-97c4-41c5-82fe-90d753135b19 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:41.992872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7714464-97c4-41c5-82fe-90d753135b19 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:41.994376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1143/2299] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:13:42.000686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8c490-f77e-4a05-8570-45e2324f1a27 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:42.002070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8c490-f77e-4a05-8570-45e2324f1a27 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:42.002582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8c490-f77e-4a05-8570-45e2324f1a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:42.023010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8bb8c490-f77e-4a05-8570-45e2324f1a27 None None] MFA Rules not processed for user `6e59c5b4dfe04f30a195ab3e83f49c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:42.024235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8bb8c490-f77e-4a05-8570-45e2324f1a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:42.031024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1157/2300] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:13:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:42.043143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7714464-97c4-41c5-82fe-90d753135b19 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:42.055400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7714464-97c4-41c5-82fe-90d753135b19 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:42.061727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:42.062715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:42.063547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:42.063547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:42.075697 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:42.077218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f2265-28b9-4c9b-8fd4-434e32fcb610 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:42.085970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1144/2301] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:42.168072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:42.168362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:42.168644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:42.170041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1158/2302] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:13:42.175628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:42.176025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:42.176438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:42.197657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:42.198945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:42.208960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1145/2303] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:42.220217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:42.222143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:42.236142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e109b-5ede-4032-9a46-982f01f9c4c0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:42.239850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:42.240480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:42.240764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:42.241038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:42.251308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:42.252730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:42.267405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1159/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:46.921796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:46.934994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee7bbf-12aa-4696-b9fe-9ea1c85f3740 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:46.941905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:46.942648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:46.943031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:46.943363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:46.945273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:47.009425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:47.011441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:47.021877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1146/2305] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:47.753914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:47.763801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f6842-2b42-4563-9c4b-bd76d0ad4f23 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:47.767946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:47.768795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:47.769302 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:47.769726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:47.772300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:47.841276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:47.842894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:47.853466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1160/2306] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:51.556374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:51.558039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:51.573287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462c2fc-359e-427b-8b8b-58d649b60ff6 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:51.578977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:51.579627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:51.579909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:51.580184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:51.582134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:51.596322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:51.598137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:51.609679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1147/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:13:51.612138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:51.613567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:51.626798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6cb92-f6f3-40ae-a38f-26dead913736 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:51.631894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:51.632850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:51.633313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:51.633652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] PATH_INFO: `/v3/users/a7ce962e4a7447848e3454905102d27c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:51.641047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7ce962e4a7447848e3454905102d27c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:51.641047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:51.688235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7ce962e4a7447848e3454905102d27c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:51.697723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7ce962e4a7447848e3454905102d27c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:51.698478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7ce962e4a7447848e3454905102d27c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:51.709999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7ce962e4a7447848e3454905102d27c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:51.710662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1e3c650b-e945-414e-8a7d-83205504ffc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7ce962e4a7447848e3454905102d27c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:51.725519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1161/2308] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:13:51 2022] DELETE /identity/v3/users/a7ce962e4a7447848e3454905102d27c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:51.907923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:51.918914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d9eb-7c1e-4287-b59f-c36ff95facbc service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:51.985344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:51.986220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:51.986537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:51.986816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] PATH_INFO: `/v3/projects/792f217e834c4049b5b0d856cd53f947` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:51.988038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=792f217e834c4049b5b0d856cd53f947)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:51.990128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.051222 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792f217e834c4049b5b0d856cd53f947', 'request_id': 'req-90a85bfe-e6be-4c85-9a29-683c5985c4c7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:52.060770 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792f217e834c4049b5b0d856cd53f947', 'request_id': 'req-90a85bfe-e6be-4c85-9a29-683c5985c4c7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:52.067323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792f217e834c4049b5b0d856cd53f947', 'request_id': 'req-90a85bfe-e6be-4c85-9a29-683c5985c4c7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:52.072028 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:52.089180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] The token cache is being invalidate because project 792f217e834c4049b5b0d856cd53f947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:13:52.089537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:52.090469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-90a85bfe-e6be-4c85-9a29-683c5985c4c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:52.092048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1148/2309] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:13:51 2022] DELETE /identity/v3/projects/792f217e834c4049b5b0d856cd53f947 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:52.098736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.099125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.099414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.118095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:52.119360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:52.163731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:52.183980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1162/2310] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:52.192473 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.192696 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.192969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.211868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:52.213089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:52.222626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1149/2311] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:52.227065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.237437 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea694de7-17b2-43e1-bc45-e5aa8a312ec8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.243447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.244101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.244376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.244642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.245403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:13:52.245699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.247330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.259706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1163/2312] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:13:52.264528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.274299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3b4c1-b5a9-444b-8363-a03bf99c19b7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.280192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.280858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.281132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.281402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.283435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.286711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.327644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1150/2313] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:13:52 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:52.331907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.342050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310e281-0468-4cc8-8933-11e6e430b550 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.402154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.402875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.403150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.403422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.404114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.406030 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.427836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1164/2314] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:13:52 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:52.432078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.443016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac123b-cb0f-4611-817f-06fad58ccaa8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.448856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.449521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.449811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.450066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.450818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.452427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.462750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1151/2315] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:52.466911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.477888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c7c44-4a5c-4296-a839-03e2d606bdd1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.483276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.483949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.484232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.484505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.485237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.486899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9e76f-3806-4bb1-ac96-683cfb105d75 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.498355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1165/2316] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:13:52.502801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.514205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545bac-63e6-465c-812e-2e3c50eb699d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.519418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.520326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.520574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.520877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] PATH_INFO: `/v3/projects/0913fc7f3aff420eb9ac74eb392acbce/users/721ca279ae0645b3aef864dd636c2d62/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.537049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0913fc7f3aff420eb9ac74eb392acbce, user_id=721ca279ae0645b3aef864dd636c2d62, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.539793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.554918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1152/2317] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:52 2022] PUT /identity/v3/projects/0913fc7f3aff420eb9ac74eb392acbce/users/721ca279ae0645b3aef864dd636c2d62/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:52.560377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2ae58-5b12-4d76-b46f-3275cf257ad9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.560669 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2ae58-5b12-4d76-b46f-3275cf257ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.561005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf2ae58-5b12-4d76-b46f-3275cf257ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.580330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8cf2ae58-5b12-4d76-b46f-3275cf257ad9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:52.581121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8cf2ae58-5b12-4d76-b46f-3275cf257ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:52.641711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1166/2318] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:52.653753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:52.655735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:52.713365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a223c-8c8c-4002-b9e0-2dd69f96f850 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:52.717957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:52.719002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:52.719276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:52.719573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:52.729324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:52.731174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:52.740825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1153/2319] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:53.485536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.485826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.486112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.513748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] MFA Rules not processed for user `721ca279ae0645b3aef864dd636c2d62`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:53.515001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:53.559272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:53.585281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1167/2320] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:53.590481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.600991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4108e-c76e-4c9a-a7f3-80cc08c9ea41 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.606208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.606903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.607200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.607443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.608133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.610049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.641457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1154/2321] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:13:53 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:53.642714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.654060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcfc65-e2b8-4568-baf1-3667d50a7584 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.714086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.714873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.715153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.715471 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.716167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.718057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.739943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1168/2322] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:13:53 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:53.743962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.754533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c7604-75c1-4c83-a478-54ba9081cbc0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.759752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.760420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.760801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.761145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.761802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.763548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.774458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1155/2323] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:53.778840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.788726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b5de-8c65-4ee7-9ecd-dd123ddd9226 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.794199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.795101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.795383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.795637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] PATH_INFO: `/v3/projects/94d98aa8b21c476ba98fe149cba7e55e/users/110ce8e2551e445aa9b7de02f54a9544/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.813522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d98aa8b21c476ba98fe149cba7e55e, user_id=110ce8e2551e445aa9b7de02f54a9544, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.816533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.832550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1169/2324] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:53 2022] PUT /identity/v3/projects/94d98aa8b21c476ba98fe149cba7e55e/users/110ce8e2551e445aa9b7de02f54a9544/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:13:53.836565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.847699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52893eb9-5d2e-4755-8d06-8b2f3a233fa0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.895887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:53.907892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.908550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.908828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.909098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.909819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.911459 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d407a6-4429-46b5-93ac-042ab4c20285 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.921854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1156/2325] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:53.926028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:53.936939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1b2b2-ddaa-41f9-ae78-e0206d0ec660 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:53.941801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:53.942730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:53.943005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:53.943275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] PATH_INFO: `/v3/projects/94d98aa8b21c476ba98fe149cba7e55e/users/110ce8e2551e445aa9b7de02f54a9544/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:53.945282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94d98aa8b21c476ba98fe149cba7e55e, user_id=110ce8e2551e445aa9b7de02f54a9544, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:53.947969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:53.958543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1170/2326] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:53 2022] PUT /identity/v3/projects/94d98aa8b21c476ba98fe149cba7e55e/users/110ce8e2551e445aa9b7de02f54a9544/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:54.577071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:54.577361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:54.577636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:54.607260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] MFA Rules not processed for user `110ce8e2551e445aa9b7de02f54a9544`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:54.608507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:54.683554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1157/2327] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:13:55.764235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:55.832467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b293f7-237f-46d3-86af-17075efc86d5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:55.837246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:55.838168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:55.838482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:55.838767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] PATH_INFO: `/v3/users/721ca279ae0645b3aef864dd636c2d62` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:55.840320 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] RBAC: Authorizing `identity:delete_user(user_id=721ca279ae0645b3aef864dd636c2d62)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:55.842314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:55.885875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:55.896208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721ca279ae0645b3aef864dd636c2d62'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:55.904767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721ca279ae0645b3aef864dd636c2d62'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:55.905583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721ca279ae0645b3aef864dd636c2d62'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:55.918402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721ca279ae0645b3aef864dd636c2d62'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:55.919088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721ca279ae0645b3aef864dd636c2d62'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:55.933266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1171/2328] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:13:55 2022] DELETE /identity/v3/users/721ca279ae0645b3aef864dd636c2d62 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:56.108509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:56.120259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08a9a4-3b18-4ea9-b60b-2b9faeb618e8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:56.180216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:56.181020 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.181297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.181592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] PATH_INFO: `/v3/projects/0913fc7f3aff420eb9ac74eb392acbce` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.182926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0913fc7f3aff420eb9ac74eb392acbce)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:56.184852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:56.233854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0913fc7f3aff420eb9ac74eb392acbce', 'request_id': 'req-2174dc5d-8301-4220-a29e-074c497799f9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.242725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0913fc7f3aff420eb9ac74eb392acbce', 'request_id': 'req-2174dc5d-8301-4220-a29e-074c497799f9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.249637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0913fc7f3aff420eb9ac74eb392acbce', 'request_id': 'req-2174dc5d-8301-4220-a29e-074c497799f9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.260290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] The token cache is being invalidate because project 0913fc7f3aff420eb9ac74eb392acbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:13:56.260597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.261905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.262814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1158/2329] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:13:56 2022] DELETE /identity/v3/projects/0913fc7f3aff420eb9ac74eb392acbce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:56.267640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:56.269083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:56.282487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627dbe3e-8a16-48c6-baa7-54b9b6e1afca admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:56.344764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:56.345637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.345914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.346182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] PATH_INFO: `/v3/users/110ce8e2551e445aa9b7de02f54a9544` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.347439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] RBAC: Authorizing `identity:delete_user(user_id=110ce8e2551e445aa9b7de02f54a9544)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:56.349366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:56.400469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110ce8e2551e445aa9b7de02f54a9544'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.409570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110ce8e2551e445aa9b7de02f54a9544'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.410199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110ce8e2551e445aa9b7de02f54a9544'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.422405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110ce8e2551e445aa9b7de02f54a9544'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.423039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-c3a1cca7-f616-4ff0-a51c-c6363852d18d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110ce8e2551e445aa9b7de02f54a9544'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.435043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1172/2330] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:13:56 2022] DELETE /identity/v3/users/110ce8e2551e445aa9b7de02f54a9544 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:56.617651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:56.619072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:56.629235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:56.633765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174dc5d-8301-4220-a29e-074c497799f9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:56.694531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:56.695411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.695691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.695961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] PATH_INFO: `/v3/projects/94d98aa8b21c476ba98fe149cba7e55e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.697191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94d98aa8b21c476ba98fe149cba7e55e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:56.699271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:56.750115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94d98aa8b21c476ba98fe149cba7e55e', 'request_id': 'req-346b1a46-6438-4ff4-ae6e-584fd2dfe652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.759119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94d98aa8b21c476ba98fe149cba7e55e', 'request_id': 'req-346b1a46-6438-4ff4-ae6e-584fd2dfe652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.765667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94d98aa8b21c476ba98fe149cba7e55e', 'request_id': 'req-346b1a46-6438-4ff4-ae6e-584fd2dfe652'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.776476 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] The token cache is being invalidate because project 94d98aa8b21c476ba98fe149cba7e55e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:13:56.776763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.777602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-346b1a46-6438-4ff4-ae6e-584fd2dfe652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:13:56.779232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1159/2331] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:13:56 2022] DELETE /identity/v3/projects/94d98aa8b21c476ba98fe149cba7e55e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:56.784421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.784850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.784971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.816010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:56.817363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:56.850065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:56.882720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1173/2332] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:56.892177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.892465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.892740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.911378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:56.912619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:56.921917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1160/2333] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:56.927021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:56.939239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887741a-c3ac-4340-9f41-540d1cfcc1ec None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:56.948197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:56.949823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.950144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.950504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.951703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:13:56.952170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:56.954579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:56.968173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1174/2334] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:13:56.973303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:56.983027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1fc0dd-625f-433f-a6e3-7ecaee896801 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:56.987994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:56.988645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:56.988924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:56.989221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:56.989898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:56.991832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.019513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1161/2335] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:13:56 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:13:57.024099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.034617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4990792-f46a-41e7-8b6c-85cfc181aa85 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.095585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.096309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.096620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.096891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.097591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.099740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.127338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1175/2336] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:13:57.131764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.142728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f266d-fcc9-40f5-9f74-6c4c421b162f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.147517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.148186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.148453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.148719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.149449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.151092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.162101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1162/2337] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:57.166434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.177049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df18d3f-46eb-47fd-b506-3c9203fe83e7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.182751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.183537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.183818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.184095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.184833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.186468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d0b9b-72f5-4078-ada3-53e6e57860b2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.197170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1176/2338] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:57.201838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.212241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f755f2-85e7-47ba-849d-c48f56bdd4f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.218302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.219608 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.219885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.220166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] PATH_INFO: `/v3/projects/deb3291c69b94f058556e68d55790c13/users/a621ed0cfcc74956bd430363b50467e1/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.236992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb3291c69b94f058556e68d55790c13, user_id=a621ed0cfcc74956bd430363b50467e1, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.239763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.263704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1163/2339] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:57 2022] PUT /identity/v3/projects/deb3291c69b94f058556e68d55790c13/users/a621ed0cfcc74956bd430363b50467e1/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:13:57.282568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.283008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.283363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.313188 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] MFA Rules not processed for user `a621ed0cfcc74956bd430363b50467e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:57.314560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:57.381358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1177/2340] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:57.386181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.421970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:57.458307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec83a60-a3ac-430b-92da-292f003907aa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.463257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.463943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.464225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.464616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.465297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.467285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.500868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1164/2341] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:57.502205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.513913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00b0fd-3f05-447b-8ab7-dd6deabddea3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.542232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:57.574190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.574907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.575189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.575453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.576151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.578084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.601221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1178/2342] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:57.605761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.616518 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1c2b64-2ba0-456f-87e2-502341368947 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.622223 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.622909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.623188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.623458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.624235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.625854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.637378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1165/2343] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:57.643355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.654216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd06782-526d-401a-bd67-1e36bed6062b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.659352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.660005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.660286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.660685 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.661427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.663087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b2b5d-1ffd-4c97-ba9f-db4719a6c0aa admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.673592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1179/2344] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:57.679003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.689325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79dbe94-b85f-4a85-a8b2-d3f2501f87ee admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.694988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.695869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.696141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.696535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] PATH_INFO: `/v3/projects/b67a3f95eea34735a778633dde88d2a1/users/7d8cc5edc80f4ab0956db7a4e566829f/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.712034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67a3f95eea34735a778633dde88d2a1, user_id=7d8cc5edc80f4ab0956db7a4e566829f, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.714865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.730624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1166/2345] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:57 2022] PUT /identity/v3/projects/b67a3f95eea34735a778633dde88d2a1/users/7d8cc5edc80f4ab0956db7a4e566829f/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:57.741899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.742186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.742503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.770340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] MFA Rules not processed for user `7d8cc5edc80f4ab0956db7a4e566829f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:57.771594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:57.842304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1180/2346] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:57.847234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.910494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82566ba8-1847-41e7-93bd-b2ddfc317ef0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:57.915958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:57.916625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:57.916899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:57.917216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:57.917869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:57.919828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:57.949445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1167/2347] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:57.953624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:57.964318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d92c3f-e6b0-45a6-9444-5420e9ebbe13 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.025325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.025959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.026614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.026909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.027624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.029693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.054363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1181/2348] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:13:57 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:13:58.056145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:58.066278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d666c4d-76a8-42db-ba89-f939c7a975d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.071197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.071855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.072137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.072418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.073234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.074925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.086030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1168/2349] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:58.090529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:58.101488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8407a59-4f34-4a59-a526-b30943505fc2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.106540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.107557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.107839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.108109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] PATH_INFO: `/v3/projects/53c4c978928b4ff1be49cbd7eee3938c/users/6aee7316935a4d1a8cb8d29d874b1029/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.123337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c4c978928b4ff1be49cbd7eee3938c, user_id=6aee7316935a4d1a8cb8d29d874b1029, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.126018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.140707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1182/2350] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:58 2022] PUT /identity/v3/projects/53c4c978928b4ff1be49cbd7eee3938c/users/6aee7316935a4d1a8cb8d29d874b1029/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:13:58.144737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:58.154802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c6b5-0eb2-43fd-88b9-938e5c73b939 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.207179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:58.215001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.215839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.216337 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.216827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.217636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.219662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a62b8-6df8-498d-b7d6-f74cc9852ba2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.229481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1169/2351] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:13:58.234044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:58.239507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:13:58.245135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f301415-39c1-49ac-9c75-734b99d12722 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.250725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.252004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.252712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.253287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] PATH_INFO: `/v3/projects/53c4c978928b4ff1be49cbd7eee3938c/users/6aee7316935a4d1a8cb8d29d874b1029/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.256296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c4c978928b4ff1be49cbd7eee3938c, user_id=6aee7316935a4d1a8cb8d29d874b1029, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.259356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.272534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1183/2352] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:13:58 2022] PUT /identity/v3/projects/53c4c978928b4ff1be49cbd7eee3938c/users/6aee7316935a4d1a8cb8d29d874b1029/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:13:58.283262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.283558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.283835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.310912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] MFA Rules not processed for user `6aee7316935a4d1a8cb8d29d874b1029`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:13:58.312124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:58.382525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1170/2353] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:13:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:13:58.393966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:58.396114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:58.460176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7886f33-098b-4280-8870-e97a5282bb8d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:58.465647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:58.466236 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:58.466549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:58.466825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:58.477284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:58.478760 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:58.488432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1184/2354] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:13:59.022655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:13:59.025072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:13:59.044892 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c676d98-8fd1-47df-ba6f-b475415a1cde None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:13:59.048867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:13:59.049528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:13:59.049808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:13:59.050111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:13:59.112423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:13:59.113886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:13:59.123081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1171/2355] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:13:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:03.190118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:03.199247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849fed0e-79d2-4d4f-9918-aa1962d14421 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:03.203435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:03.204073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:03.204351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:03.204621 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:03.262032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:03.263362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf08bd-d021-4c11-87b9-30866d57304e service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:03.272996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1185/2356] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:05.443526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:05.508598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189dc2a-5eaa-44cf-86d0-09a99e9fd6ca service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:05.514361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:05.515252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:05.515531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:05.515802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] PATH_INFO: `/v3/users/a621ed0cfcc74956bd430363b50467e1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:05.517030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a621ed0cfcc74956bd430363b50467e1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:05.518992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:05.535805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:05.570606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a621ed0cfcc74956bd430363b50467e1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:05.580142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a621ed0cfcc74956bd430363b50467e1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:05.581063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a621ed0cfcc74956bd430363b50467e1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:05.601050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a621ed0cfcc74956bd430363b50467e1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:05.602064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a621ed0cfcc74956bd430363b50467e1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:05.620287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1172/2357] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:05 2022] DELETE /identity/v3/users/a621ed0cfcc74956bd430363b50467e1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:05.622217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:05.622730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:05.622826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:05.640497 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:05.641605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:05.702452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1186/2358] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:05.712487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:05.713936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:05.772441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193dda-cc64-4dfc-aba5-0579a14b059c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:05.777051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:05.777706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:05.777986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:05.778264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:05.788783 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:05.790267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:05.800251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1173/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:06.075342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:06.076869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:06.089999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d6f79-7155-4758-a36b-55fe42a7dc48 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:06.096459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:06.097357 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:06.097640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:06.097914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] PATH_INFO: `/v3/projects/deb3291c69b94f058556e68d55790c13` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:06.099392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb3291c69b94f058556e68d55790c13)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:06.101370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:06.135242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:06.152648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb3291c69b94f058556e68d55790c13', 'request_id': 'req-248eea16-b275-4091-b149-3349b57afe48'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.163248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb3291c69b94f058556e68d55790c13', 'request_id': 'req-248eea16-b275-4091-b149-3349b57afe48'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.169593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb3291c69b94f058556e68d55790c13', 'request_id': 'req-248eea16-b275-4091-b149-3349b57afe48'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.181303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] The token cache is being invalidate because project deb3291c69b94f058556e68d55790c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:06.181699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.182232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.183891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1187/2360] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:06 2022] DELETE /identity/v3/projects/deb3291c69b94f058556e68d55790c13 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:06.188142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:06.198085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a56c7-f953-4c08-95ba-b805d43bd179 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:06.253241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:06.254111 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:06.254422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:06.254698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] PATH_INFO: `/v3/users/7d8cc5edc80f4ab0956db7a4e566829f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:06.255918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d8cc5edc80f4ab0956db7a4e566829f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:06.257856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:06.302455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:06.309572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d8cc5edc80f4ab0956db7a4e566829f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.320217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d8cc5edc80f4ab0956db7a4e566829f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.320890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d8cc5edc80f4ab0956db7a4e566829f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.333894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d8cc5edc80f4ab0956db7a4e566829f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.334568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d8cc5edc80f4ab0956db7a4e566829f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.348606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1174/2361] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:06 2022] DELETE /identity/v3/users/7d8cc5edc80f4ab0956db7a4e566829f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:06.631843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:06.633409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:06.646844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248eea16-b275-4091-b149-3349b57afe48 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:06.708501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:06.709294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:06.709594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:06.709866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] PATH_INFO: `/v3/projects/b67a3f95eea34735a778633dde88d2a1` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:06.711240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67a3f95eea34735a778633dde88d2a1)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:06.713222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:06.762349 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67a3f95eea34735a778633dde88d2a1', 'request_id': 'req-1b9c6653-1d45-41a4-b79d-fba75ae785b5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.770993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67a3f95eea34735a778633dde88d2a1', 'request_id': 'req-1b9c6653-1d45-41a4-b79d-fba75ae785b5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.777239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67a3f95eea34735a778633dde88d2a1', 'request_id': 'req-1b9c6653-1d45-41a4-b79d-fba75ae785b5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.788082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] The token cache is being invalidate because project b67a3f95eea34735a778633dde88d2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:06.788366 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.789071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.790679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1188/2362] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:06 2022] DELETE /identity/v3/projects/b67a3f95eea34735a778633dde88d2a1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:06.794930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:06.796363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:06.808995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d9cfe-496e-484c-8e3d-c6b6872940e7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:06.871955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:06.872824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:06.873181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:06.873458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] PATH_INFO: `/v3/users/6aee7316935a4d1a8cb8d29d874b1029` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:06.874731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aee7316935a4d1a8cb8d29d874b1029)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:06.876652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:06.937502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aee7316935a4d1a8cb8d29d874b1029'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.947559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aee7316935a4d1a8cb8d29d874b1029'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.948328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aee7316935a4d1a8cb8d29d874b1029'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.960738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aee7316935a4d1a8cb8d29d874b1029'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.961304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa871da7-8fee-41c5-a702-b7df4852d77c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aee7316935a4d1a8cb8d29d874b1029'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:06.975442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1175/2363] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:06 2022] DELETE /identity/v3/users/6aee7316935a4d1a8cb8d29d874b1029 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:07.251713 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.253256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:07.270994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c6653-1d45-41a4-b79d-fba75ae785b5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.313920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:07.334055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.335340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.335732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.336114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] PATH_INFO: `/v3/projects/53c4c978928b4ff1be49cbd7eee3938c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.337904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c4c978928b4ff1be49cbd7eee3938c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.341014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.404322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c4c978928b4ff1be49cbd7eee3938c', 'request_id': 'req-a079d35c-1fb9-484f-b0f6-cc83623de99b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:07.414153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c4c978928b4ff1be49cbd7eee3938c', 'request_id': 'req-a079d35c-1fb9-484f-b0f6-cc83623de99b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:07.424617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c4c978928b4ff1be49cbd7eee3938c', 'request_id': 'req-a079d35c-1fb9-484f-b0f6-cc83623de99b'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:07.438726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] The token cache is being invalidate because project 53c4c978928b4ff1be49cbd7eee3938c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:07.439011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:07.439775 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a079d35c-1fb9-484f-b0f6-cc83623de99b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:07.441461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1189/2364] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:07 2022] DELETE /identity/v3/projects/53c4c978928b4ff1be49cbd7eee3938c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:07.448444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.448813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.449250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.481026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:07.481900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:07.520228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:07.556316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1176/2365] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:07.566736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.567026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.567300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.585618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:07.586847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:07.595624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1190/2366] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:07.599919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.609741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0bba1-8119-472e-b969-acfa849bc0f0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.614527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.615173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.615453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.615723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.616446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:14:07.616747 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.618350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.630121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1177/2367] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:14:07.635151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.649540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50140d32-2a8c-4bf9-b46e-2cefe7d38812 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.657386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.658032 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.658336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.658624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.659341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.661246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.689767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1191/2368] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:07 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:07.694131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.704670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a69a1a2-0f30-4371-9b7c-52589df74649 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.771192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.771774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.772056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.772340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.773033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.775018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.798711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1178/2369] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:07 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:07.802860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.813307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fd8b7-c63d-4a88-88b0-303a9708b150 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.818493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.819148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.819438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.819717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.820440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.822272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.839869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1192/2370] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:07.845776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.858053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970ce33-8fe2-49b9-aa84-3b4a86c85ddd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.863995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.864656 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.864938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.865221 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.865944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.867605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c59de1-2460-44ea-9739-018f34885031 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.881009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1179/2371] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:07.886157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:07.897688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6704c-33cf-4f35-9048-e12f8853d75d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:07.903784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:07.904720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.904990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.905262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] PATH_INFO: `/v3/projects/384295bc6c124d19a7c2030d83ba8f62/users/7cb499c94f0948e89468c3f4df377a20/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.920945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=384295bc6c124d19a7c2030d83ba8f62, user_id=7cb499c94f0948e89468c3f4df377a20, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:07.923716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:07.939132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1193/2372] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:07 2022] PUT /identity/v3/projects/384295bc6c124d19a7c2030d83ba8f62/users/7cb499c94f0948e89468c3f4df377a20/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:07.952680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:07.953061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:07.953374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:07.983140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] MFA Rules not processed for user `7cb499c94f0948e89468c3f4df377a20`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:07.984384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:08.057199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1180/2373] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:08.061684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.121552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3398583-9ec0-448b-83ae-354a4cfd1d2c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.126899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.127548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.127828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.128095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.128767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.130694 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.145228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:08.162255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1194/2374] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:08 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:08.167093 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.184335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3aafa-4bfd-4706-96c4-41ad1093f7d0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.215083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:08.246129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.247457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.247457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.247457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.248389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.250841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.276199 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1181/2375] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:08 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:08.280346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.290939 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4a4f96-83a7-4dce-bedf-b69eb94db013 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.296559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.297213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.297492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.297780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.298518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.300114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.310746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1195/2376] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:08.315124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.325992 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed195a-e481-40b7-b25c-aab091c15529 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.331883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.332780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.333063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.333341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] PATH_INFO: `/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf/users/1d3a63a1337e4cecb30fb6dcf9f3940f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.350057 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f214657b3d4aa28afe87fee6e4eecf, user_id=1d3a63a1337e4cecb30fb6dcf9f3940f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.352811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.368357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1182/2377] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:08 2022] PUT /identity/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf/users/1d3a63a1337e4cecb30fb6dcf9f3940f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:08.372487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.383147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0505cd-a8b4-4c0f-a2b1-00866ea55b33 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.444182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.444836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.445116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.445386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.446078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.447709 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4f4dc-f6c2-4262-9e5e-b25a5fd3c46b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.457582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1196/2378] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:08.461829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.473361 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35dde5-799d-4b7a-ba65-08cfc18446aa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.480240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.481614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.482002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.482433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] PATH_INFO: `/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf/users/1d3a63a1337e4cecb30fb6dcf9f3940f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.484807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f214657b3d4aa28afe87fee6e4eecf, user_id=1d3a63a1337e4cecb30fb6dcf9f3940f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.487552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.499604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1183/2379] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:08 2022] PUT /identity/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf/users/1d3a63a1337e4cecb30fb6dcf9f3940f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:08.517731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.518024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.518301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.547253 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] MFA Rules not processed for user `1d3a63a1337e4cecb30fb6dcf9f3940f`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:08.548462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:08.622108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1197/2380] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:14:08.634217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.637328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:08.697888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59c1f5-1c2f-4c63-a8d9-d920c7250490 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.702285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.702949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.703232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.703501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.713103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.714604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.728447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1184/2381] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:08.777874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:08.779448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:08.792677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76565-4398-476e-bd9e-3e6f808d1755 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:08.797092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:08.797724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:08.798002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:08.798270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:08.858850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:08.860310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318aa07-218c-425f-8e91-9f0a5074a0b5 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:08.869972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1198/2382] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:12.206590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:12.272652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4b840-a6ee-4017-9875-a83a7cb73570 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:12.278742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:12.279610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:12.279943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:12.280144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] PATH_INFO: `/v3/users/7cb499c94f0948e89468c3f4df377a20` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:12.281432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb499c94f0948e89468c3f4df377a20)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:12.283447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:12.315781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:12.336615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb499c94f0948e89468c3f4df377a20'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.346419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb499c94f0948e89468c3f4df377a20'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.347037 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb499c94f0948e89468c3f4df377a20'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.359005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb499c94f0948e89468c3f4df377a20'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.359702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb499c94f0948e89468c3f4df377a20'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.372620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1185/2383] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:12 2022] DELETE /identity/v3/users/7cb499c94f0948e89468c3f4df377a20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:12.377403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:12.377882 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:12.377986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:12.384549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:12.395782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:12.397002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:12.461315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1199/2384] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:14:12.472112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:12.473641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:12.545571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd58b66-c98c-4781-b42d-f40fe43be827 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:12.550119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:12.550801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:12.551084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:12.551352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:12.562286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:12.563773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:12.573781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1186/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:14:12.845852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:12.847550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:12.860965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e8f4f-e715-4d2e-91ed-cbed7a1baa66 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:12.866498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:12.867408 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:12.867667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:12.867969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] PATH_INFO: `/v3/projects/384295bc6c124d19a7c2030d83ba8f62` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:12.869315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384295bc6c124d19a7c2030d83ba8f62)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:12.871324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:12.928985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384295bc6c124d19a7c2030d83ba8f62', 'request_id': 'req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.939489 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384295bc6c124d19a7c2030d83ba8f62', 'request_id': 'req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.947455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384295bc6c124d19a7c2030d83ba8f62', 'request_id': 'req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.959934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] The token cache is being invalidate because project 384295bc6c124d19a7c2030d83ba8f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:12.960235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.960892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:12.962513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1200/2386] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:12 2022] DELETE /identity/v3/projects/384295bc6c124d19a7c2030d83ba8f62 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:12.966799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:12.977515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d5e3-e88d-487b-a719-b589ba01b996 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.037338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.038215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.038525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.038794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] PATH_INFO: `/v3/users/1d3a63a1337e4cecb30fb6dcf9f3940f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.039965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d3a63a1337e4cecb30fb6dcf9f3940f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.041886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.102635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d3a63a1337e4cecb30fb6dcf9f3940f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.106417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:13.114022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d3a63a1337e4cecb30fb6dcf9f3940f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.115121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d3a63a1337e4cecb30fb6dcf9f3940f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.127562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d3a63a1337e4cecb30fb6dcf9f3940f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.128206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-78a24bf0-0f5a-4a95-887a-76ae4f4b4102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d3a63a1337e4cecb30fb6dcf9f3940f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.140285 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1187/2387] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:12 2022] DELETE /identity/v3/users/1d3a63a1337e4cecb30fb6dcf9f3940f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:14:13.416300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.417910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:13.435737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b519c-01e8-44cf-ab3a-2c2e4998fd28 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.493728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.494620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.494895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.495166 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] PATH_INFO: `/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.496386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f214657b3d4aa28afe87fee6e4eecf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.498545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.535246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:13.550052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f214657b3d4aa28afe87fee6e4eecf', 'request_id': 'req-b6ad13bb-76d6-401d-a1c6-455d92477b13'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.559022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f214657b3d4aa28afe87fee6e4eecf', 'request_id': 'req-b6ad13bb-76d6-401d-a1c6-455d92477b13'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.565769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f214657b3d4aa28afe87fee6e4eecf', 'request_id': 'req-b6ad13bb-76d6-401d-a1c6-455d92477b13'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.577622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] The token cache is being invalidate because project b9f214657b3d4aa28afe87fee6e4eecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:13.577914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.578662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-b6ad13bb-76d6-401d-a1c6-455d92477b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:13.580519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1201/2388] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:13 2022] DELETE /identity/v3/projects/b9f214657b3d4aa28afe87fee6e4eecf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:13.586723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.587242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.587242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.607401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:13.608680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:13.671414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1188/2389] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:13.679539 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.679830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.680108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.701284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:13.703101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:13.711951 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1202/2390] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:13.716381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.726662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e190-5d96-464e-bf9d-8ef7dfcbaf74 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.731870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.732635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.732938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.733205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.733921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:14:13.734228 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.735865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.747576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1189/2391] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:14:13.752199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.761811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff8a82-1e0b-45e4-82f0-d296790df65a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.768240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.768890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.769167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.769439 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.770132 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.772098 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.803929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1203/2392] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:13 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:13.809814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.820216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb4eea-d06a-4a57-bd2c-6f912705a141 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.880605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.881225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.881502 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.881773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.882467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.884442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.906124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1190/2393] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:13 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:13.910487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.921279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478e90a-8934-44a7-9084-262092fcad2d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.926668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.927352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.927630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.927901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.928656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.930246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.941414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1204/2394] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:13.945585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.956103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050ad43-338c-4c51-8fe5-848f0b6469f7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.961179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.961836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.962116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.962418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:13.963147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:13.964748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaadc0f-2d2b-47f3-aaa2-3181c4dbebf2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:13.975187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1191/2395] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:13.979393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:13.990160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5549bd-3d38-435b-9a2b-5d9e452fa5e3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:13.996126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:13.997045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:13.997327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:13.997602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] PATH_INFO: `/v3/projects/5afcb7860cab4620ba719e5a9bc79d41/users/82718cc94628460aafcf0ebec3a073bb/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.014825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afcb7860cab4620ba719e5a9bc79d41, user_id=82718cc94628460aafcf0ebec3a073bb, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.017485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.033337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1205/2396] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:13 2022] PUT /identity/v3/projects/5afcb7860cab4620ba719e5a9bc79d41/users/82718cc94628460aafcf0ebec3a073bb/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:14.101023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.101421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.101894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.137733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] MFA Rules not processed for user `82718cc94628460aafcf0ebec3a073bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:14.138978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:14.160592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:14.207491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1192/2397] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:14.212218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.288323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95aa39-d761-49b6-89e6-541b16682a71 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.293584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.294214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.294518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.294853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.295681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.297595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.325899 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1206/2398] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:14.330131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.341192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6d263-abdb-482c-b1ba-9aa8d4b6972b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.407022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.407605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.407880 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.408148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.408835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.410790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.434800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1193/2399] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:14.439333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.449879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4b1a-6347-4b0d-9530-64bcf63c31d6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.455078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.455741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.456015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.456700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.457464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.458634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.470237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1207/2400] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:14.474617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.485604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87c9c5-2234-4d35-bc5e-ba63cbd8bb72 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.491692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.492341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.492617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.492885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.493613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.495255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc41f01-8ad7-4489-a994-9fdb9d236328 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.507585 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1194/2401] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:14.511878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.522147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bee63-46b6-4cc1-9a19-6a10e1b83da8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.527044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.527913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.528193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.528462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] PATH_INFO: `/v3/projects/ba6bf3f12d444d25a7474bedda6c25d5/users/cb69ec07615740a5954aaec983300f3d/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.543397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba6bf3f12d444d25a7474bedda6c25d5, user_id=cb69ec07615740a5954aaec983300f3d, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.546185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.561673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1208/2402] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:14 2022] PUT /identity/v3/projects/ba6bf3f12d444d25a7474bedda6c25d5/users/cb69ec07615740a5954aaec983300f3d/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:14.572050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.572342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.572625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.599555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] MFA Rules not processed for user `cb69ec07615740a5954aaec983300f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:14.600785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:14.669206 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1195/2403] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:14.674524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.681486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:14.744536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89470a-aea6-4c16-92b1-1495982bdcda admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.750912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.751577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.751855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.752122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.752816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.754767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.783613 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1209/2404] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:14.787940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.799528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416cd21-ad08-46ef-abff-de5b1628d75b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.831982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:14.862420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.863092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.863374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.863647 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.864333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.866260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.888386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1196/2405] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:14 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:14.892847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.905919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a5f35-7e28-48b7-8021-32b4da3acda7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.912145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.912789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.913064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.913329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.914050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.915690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.926772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1210/2406] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:14.931094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.942017 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069b81b-fc0e-4507-b3d4-164f4dcd2320 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:14.947651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:14.948530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:14.948805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:14.949074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] PATH_INFO: `/v3/projects/5cd84f796f56467b967b6d741bf82a49/users/fbdb2b3f18c147d6b5609617b3d278fd/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:14.965263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd84f796f56467b967b6d741bf82a49, user_id=fbdb2b3f18c147d6b5609617b3d278fd, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:14.968026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:14.983867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1197/2407] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:14 2022] PUT /identity/v3/projects/5cd84f796f56467b967b6d741bf82a49/users/fbdb2b3f18c147d6b5609617b3d278fd/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:14.987840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:14.998344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228797a-a717-4169-9e88-44aea39a2090 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:15.054462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:15.055130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:15.055406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:15.055679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:15.056396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:15.057994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9db1c-c7d9-414e-b7dd-4b51fdb64175 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:15.068581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1211/2408] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:15.073061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:15.084102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b32e0a-d2f8-4669-a58a-8f5a29631e0c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:15.089150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:15.090044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:15.090306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:15.090601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] PATH_INFO: `/v3/projects/5cd84f796f56467b967b6d741bf82a49/users/fbdb2b3f18c147d6b5609617b3d278fd/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:15.092768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cd84f796f56467b967b6d741bf82a49, user_id=fbdb2b3f18c147d6b5609617b3d278fd, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:15.095788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:15.107246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1198/2409] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:15 2022] PUT /identity/v3/projects/5cd84f796f56467b967b6d741bf82a49/users/fbdb2b3f18c147d6b5609617b3d278fd/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:15.119847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:15.120325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:15.120786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:15.149395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] MFA Rules not processed for user `fbdb2b3f18c147d6b5609617b3d278fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:15.150666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:15.214715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:15.218371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1212/2410] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:15.230344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:15.232466 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:15.289726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea32b4-6abb-4bff-ad5e-14e66a53f7ee admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:15.293620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:15.294257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:15.294575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:15.294864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:15.357969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:15.359477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:15.368560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1199/2411] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:16.881790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:16.883323 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:16.895912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e696b-c2f8-476e-ba80-78edb0e6be91 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:16.900111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:16.900741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:16.901024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:16.901298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:16.960906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:16.962343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:16.972002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1213/2412] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:17.537551 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:17.546544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a6611-1d9e-4bb7-bdd1-5be98c18b6d6 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:17.550494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:17.551124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:17.551392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:17.551664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:17.560918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:17.562722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374c157-1ae9-4b4e-b9b1-b8d8ef985ade service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:17.572258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1200/2413] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:21.512480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:21.578256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba414e3-ea36-4919-9c32-cf5e63e077ea service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:21.583625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:21.584521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:21.584804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:21.585077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] PATH_INFO: `/v3/users/82718cc94628460aafcf0ebec3a073bb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:21.586352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82718cc94628460aafcf0ebec3a073bb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:21.588317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:21.638015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82718cc94628460aafcf0ebec3a073bb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:21.647657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82718cc94628460aafcf0ebec3a073bb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:21.648317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82718cc94628460aafcf0ebec3a073bb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:21.659967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82718cc94628460aafcf0ebec3a073bb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:21.660587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82718cc94628460aafcf0ebec3a073bb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:21.673999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1214/2414] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:21 2022] DELETE /identity/v3/users/82718cc94628460aafcf0ebec3a073bb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:21.678796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:21.679320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:21.679320 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:21.697895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:21.699138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:21.718681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:21.759843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1201/2415] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:14:21.769806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:21.771490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:21.796424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:21.835985 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696b076b-7f4a-4122-ac2e-e0649fac2092 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:21.839763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:21.840410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:21.840690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:21.840965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:21.851045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:21.852480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:21.862712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1215/2416] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:22.391709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:22.393327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:22.407637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aecea-f335-43bb-a42f-5b48d441835d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:22.413286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:22.414577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:22.414762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:22.415051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] PATH_INFO: `/v3/projects/5afcb7860cab4620ba719e5a9bc79d41` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:22.416441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afcb7860cab4620ba719e5a9bc79d41)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:22.418387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:22.467890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afcb7860cab4620ba719e5a9bc79d41', 'request_id': 'req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.477281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afcb7860cab4620ba719e5a9bc79d41', 'request_id': 'req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.483519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afcb7860cab4620ba719e5a9bc79d41', 'request_id': 'req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.494276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] The token cache is being invalidate because project 5afcb7860cab4620ba719e5a9bc79d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:22.494589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.495230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.496794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1202/2417] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:22 2022] DELETE /identity/v3/projects/5afcb7860cab4620ba719e5a9bc79d41 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:22.500910 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:22.510835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aec13-1e95-403e-8bb6-be5944547dc3 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:22.568156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:22.569031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:22.569310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:22.569584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] PATH_INFO: `/v3/users/cb69ec07615740a5954aaec983300f3d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:22.570855 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb69ec07615740a5954aaec983300f3d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:22.572779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:22.622979 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb69ec07615740a5954aaec983300f3d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.632604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb69ec07615740a5954aaec983300f3d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.633526 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb69ec07615740a5954aaec983300f3d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.647130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb69ec07615740a5954aaec983300f3d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.647816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb69ec07615740a5954aaec983300f3d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:22.659877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1216/2418] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:22 2022] DELETE /identity/v3/users/cb69ec07615740a5954aaec983300f3d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:22.949739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:22.951481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:22.965367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1da98-75ea-47d9-a158-5e92f7d4ebdd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:23.024066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:23.024948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.025227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.025500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] PATH_INFO: `/v3/projects/ba6bf3f12d444d25a7474bedda6c25d5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.027004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba6bf3f12d444d25a7474bedda6c25d5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:23.028955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:23.078077 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba6bf3f12d444d25a7474bedda6c25d5', 'request_id': 'req-13bcd724-5983-4cd5-8a3c-ce369278523f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.079662 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:23.087672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba6bf3f12d444d25a7474bedda6c25d5', 'request_id': 'req-13bcd724-5983-4cd5-8a3c-ce369278523f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.094447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba6bf3f12d444d25a7474bedda6c25d5', 'request_id': 'req-13bcd724-5983-4cd5-8a3c-ce369278523f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.105101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] The token cache is being invalidate because project ba6bf3f12d444d25a7474bedda6c25d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:23.105387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.106082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.107706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1203/2419] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:22 2022] DELETE /identity/v3/projects/ba6bf3f12d444d25a7474bedda6c25d5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:23.119115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:23.121433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:23.141247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b7801-2c5b-4da7-8d2f-cc75e97b67ea admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:23.184083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:23.205902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:23.206829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.207107 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.207377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] PATH_INFO: `/v3/users/fbdb2b3f18c147d6b5609617b3d278fd` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.208636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbdb2b3f18c147d6b5609617b3d278fd)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:23.210606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:23.265567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbdb2b3f18c147d6b5609617b3d278fd'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.275134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbdb2b3f18c147d6b5609617b3d278fd'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.275830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbdb2b3f18c147d6b5609617b3d278fd'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.288430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbdb2b3f18c147d6b5609617b3d278fd'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.289087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a743aebd-7818-440e-9c88-60ec5ce1d91b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbdb2b3f18c147d6b5609617b3d278fd'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.302005 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1217/2420] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:14:23 2022] DELETE /identity/v3/users/fbdb2b3f18c147d6b5609617b3d278fd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:23.593690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:23.595602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:23.608877 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcd724-5983-4cd5-8a3c-ce369278523f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:23.669632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:23.670462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.670739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.671009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] PATH_INFO: `/v3/projects/5cd84f796f56467b967b6d741bf82a49` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.672306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cd84f796f56467b967b6d741bf82a49)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:23.674447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:23.725842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cd84f796f56467b967b6d741bf82a49', 'request_id': 'req-aa2509c0-7ab1-46a4-9662-fed5a906123c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.735456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cd84f796f56467b967b6d741bf82a49', 'request_id': 'req-aa2509c0-7ab1-46a4-9662-fed5a906123c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.742478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cd84f796f56467b967b6d741bf82a49', 'request_id': 'req-aa2509c0-7ab1-46a4-9662-fed5a906123c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.752946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] The token cache is being invalidate because project 5cd84f796f56467b967b6d741bf82a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:14:23.753230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.753923 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aa2509c0-7ab1-46a4-9662-fed5a906123c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:14:23.755580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1204/2421] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:14:23 2022] DELETE /identity/v3/projects/5cd84f796f56467b967b6d741bf82a49 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:14:23.760743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.761218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.761334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.779904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:23.781121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:23.840509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1218/2422] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:23.849043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.849329 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.849605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.870895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:23.872121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:23.881142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1205/2423] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:14:23.885706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:23.896057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffee010-08fb-4c3d-9534-000a5f8104a9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:23.901993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:23.902210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.902521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.902797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.903525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:14:23.903833 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:23.905531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:23.917998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1219/2424] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:14:23.922818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:23.932685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed4c60-d6af-419a-b2f7-c683bbb7c501 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:23.937622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:23.938279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:23.938572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:23.938826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:23.939538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:23.941453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:23.978239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1206/2425] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:14:23 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:23.982592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:23.993261 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04fe5f-db5b-4e4f-a66f-f49b523c1b43 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:24.039774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:24.055318 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:24.056015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.056310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.056583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.057272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:24.059222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:24.084014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1220/2426] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:14:23 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:14:24.085545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:24.096711 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1836a3-1ea5-4276-94e2-727e61f40262 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:24.102276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:24.103174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.103457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.103727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.104619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:24.106225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:24.117534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1207/2427] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:14:24.121588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:24.133092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ef6d8-0d7b-49c5-bab8-1249309571a2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:24.138387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:24.139056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.139341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.139652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.140352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:24.141969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4f201-b613-4e49-9b6b-e07b91aac92d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:24.152638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1221/2428] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:14:24.157143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:24.169080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5f7d9-a73c-4e26-a8d3-1905551aa23d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:24.175391 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:24.176820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.177268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.177703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] PATH_INFO: `/v3/projects/f513a61131494d509af5ee482017c640/users/9b49d439722a4d758769304308d29b78/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.194771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f513a61131494d509af5ee482017c640, user_id=9b49d439722a4d758769304308d29b78, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:24.197478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:24.211714 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1208/2429] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:14:24 2022] PUT /identity/v3/projects/f513a61131494d509af5ee482017c640/users/9b49d439722a4d758769304308d29b78/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:14:24.222758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.223214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.223655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.253191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] MFA Rules not processed for user `9b49d439722a4d758769304308d29b78`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:24.254533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:24.335943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1222/2430] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:24.347761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:24.349642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:24.356349 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:24.409855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec0287-b1bb-4ce3-addd-3e6803c4cae3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:24.413629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:24.414263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:24.414580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:24.414850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:24.423426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:24.424868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26130cdd-5761-45c9-b789-03e466570858 service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:24.433707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1209/2431] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:34.332784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:34.429744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=24 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:35.405880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:35.408255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:35.499707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70855c-7aaa-445e-a380-fb2ed330d8b7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:35.505176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:35.505996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:35.506444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:35.506720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:35.508688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:35.521391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:35.523160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:35.535271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1223/2432] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:14:39.006341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ed56-36d7-44b6-902e-ebd46decbce5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:39.006894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ed56-36d7-44b6-902e-ebd46decbce5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:39.006894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ed56-36d7-44b6-902e-ebd46decbce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:39.026128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f149ed56-36d7-44b6-902e-ebd46decbce5 None None] MFA Rules not processed for user `126e2b6fc7114ec3bdc1bb3a99fcf752`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:39.027377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f149ed56-36d7-44b6-902e-ebd46decbce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:39.084376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1210/2433] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:14:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:39.409464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:39.411013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:39.471001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab63f93-c1e6-4f2d-bc49-3073a34a867b service placement] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:39.475092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:39.475794 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:39.476015 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:39.476285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:39.478231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:39.488407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:39.489835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:39.498006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1224/2434] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:14:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:14:49.077072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=25 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:49.494278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=58 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:14:59.581713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:59.582219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:59.582351 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:59.605178 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:14:59.606427 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:59.669425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1211/2435] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:14:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:14:59.718269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:14:59.720160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:14:59.812473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7ed5e-6e85-44cd-8036-9762fb8ed28b service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:14:59.817905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:14:59.818832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:14:59.819160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:14:59.819507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:14:59.829944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:14:59.831866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14bc5-3bdd-41d8-8cb2-651c8f15c60a service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:14:59.840769 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1225/2436] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:14:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:15:03.050780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:03.124692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1e84d-0b37-4746-ab51-bd82bfc5e04c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:03.132303 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:03.133200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:03.133479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:03.134615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] PATH_INFO: `/v3/users/9b49d439722a4d758769304308d29b78` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:03.136134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b49d439722a4d758769304308d29b78)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:03.138380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:03.221735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b49d439722a4d758769304308d29b78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.234250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b49d439722a4d758769304308d29b78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.235065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b49d439722a4d758769304308d29b78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.249962 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b49d439722a4d758769304308d29b78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.250882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b49d439722a4d758769304308d29b78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.275309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1212/2437] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:15:03 2022] DELETE /identity/v3/users/9b49d439722a4d758769304308d29b78 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:15:03.279997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:03.280467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:03.280564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:03.306892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:03.309114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:03.380646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1226/2438] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:15:03.391496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:03.393567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:03.427143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:03.473161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633a3ee-133c-462d-8635-5700b3b63812 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:03.477225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:03.477925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:03.478209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:03.478645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:03.489686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:03.491316 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2468cf-fc5a-4180-b423-46a167c5b70e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:03.502541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1213/2439] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:03.852636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:03.854178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:03.876274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac4f7c-8e6c-41b5-98cc-88f2cfe13e66 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:03.896947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:03.897757 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:03.898044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:03.898367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] PATH_INFO: `/v3/projects/f513a61131494d509af5ee482017c640` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:03.899922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f513a61131494d509af5ee482017c640)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:03.901980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:03.969079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f513a61131494d509af5ee482017c640', 'request_id': 'req-0653f72a-7274-4345-994e-f79d7715d9b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.979138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f513a61131494d509af5ee482017c640', 'request_id': 'req-0653f72a-7274-4345-994e-f79d7715d9b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:03.989207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f513a61131494d509af5ee482017c640', 'request_id': 'req-0653f72a-7274-4345-994e-f79d7715d9b6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:04.001033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] The token cache is being invalidate because project f513a61131494d509af5ee482017c640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:15:04.001449 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:04.002118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-0653f72a-7274-4345-994e-f79d7715d9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:15:04.004002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1227/2440] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:15:03 2022] DELETE /identity/v3/projects/f513a61131494d509af5ee482017c640 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:15:04.009281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.009574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.009854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.030748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:04.032239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:04.100868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1214/2441] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:04.109144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.109663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.110119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.130129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:04.131536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:04.143153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1228/2442] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:04.144539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.156074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117980e3-1e33-4e3e-a042-97d22631e340 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.162197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.162957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.163240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.163514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.164533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:15:04.164719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.166378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.180968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1215/2443] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:15:04.185403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.195317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc8f01-de8d-4a76-a6eb-1432cc4434b6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.201770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.201770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.202240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.202652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.203457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.205635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.209661 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:04.251308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1229/2444] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:15:04 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:15:04.252568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.264589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c393d6-0afa-4a50-b272-1ab6a5db2952 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.332082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.332938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.333406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.333627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.334446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.336536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.365452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1216/2445] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:15:04 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:15:04.368675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.381839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22944867-5a5c-4e16-befc-1742d1af4edb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.390041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.390829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.391106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.391367 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.392211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.393996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.416971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1230/2446] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:15:04.418713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.430694 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0fff3-a4ac-447a-a03f-9e61feb8dd68 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.436115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.437081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.437451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.437848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.438899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.440932 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5799c0-4aed-4f9a-8f70-37b232cc575c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.454580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1217/2447] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:15:04.458673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.471422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90204f5a-4b83-42b3-8d4f-cd2c62185fa7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.477444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.478412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.478696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.478972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] PATH_INFO: `/v3/projects/80ff68a0b07e43d4bdb9c32f7166c9a1/users/1c4aa9e4d27c4c2ea4b49ab188f841da/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.495677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ff68a0b07e43d4bdb9c32f7166c9a1, user_id=1c4aa9e4d27c4c2ea4b49ab188f841da, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.498595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.517735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1231/2448] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:15:04 2022] PUT /identity/v3/projects/80ff68a0b07e43d4bdb9c32f7166c9a1/users/1c4aa9e4d27c4c2ea4b49ab188f841da/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:15:04.523150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d112-45f1-4f34-a4f4-baaefb39d188 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.523443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d112-45f1-4f34-a4f4-baaefb39d188 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.523724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288d112-45f1-4f34-a4f4-baaefb39d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.546767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3288d112-45f1-4f34-a4f4-baaefb39d188 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:04.548390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3288d112-45f1-4f34-a4f4-baaefb39d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:04.585688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:04.623377 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1218/2449] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:04.638458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:04.641960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:04.723988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d8aee2-02a3-405d-ad73-be7484c106a9 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:04.731741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:04.732706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:04.733215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:04.733705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:04.748721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:04.750337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:04.761037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1232/2450] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:10.062723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.063317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.063317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.128555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] MFA Rules not processed for user `1c4aa9e4d27c4c2ea4b49ab188f841da`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:10.133277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:10.218681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1219/2451] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:10.223562 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.234480 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f83571-0831-4eb0-90cd-d026f4426dc3 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.240743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.241422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.241704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.241981 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.242732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.244744 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.288851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1233/2452] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:15:10 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:15:10.293690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.306241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e08e7-d240-4ac0-931a-26777a2ca48d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.367870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.368569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.368856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.369130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.370076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.372224 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.396153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1220/2453] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:15:10 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:15:10.400245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.411872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee22407-bbb8-4cc9-a407-56e9917bbed8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.416708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.417676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.417963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.418244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.419122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.420843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.435667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1234/2454] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:15:10.436842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.447360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360ee6d8-bd46-440e-bd7e-616a38b1a4b8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.452278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.453259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.453624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.453955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] PATH_INFO: `/v3/projects/44f7c71ddce64c24acf7eea84534609b/users/2eed55ce903b45628fc28a9e747b235d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.470672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f7c71ddce64c24acf7eea84534609b, user_id=2eed55ce903b45628fc28a9e747b235d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.473394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.489995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1221/2455] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:15:10 2022] PUT /identity/v3/projects/44f7c71ddce64c24acf7eea84534609b/users/2eed55ce903b45628fc28a9e747b235d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:15:10.494112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.505726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4accef-cb4f-4eb7-aeaf-93c0f931d7c7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.576978 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.586198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.586198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.586198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.586198 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.587820 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a939b-f922-425b-ab05-8ebe38445c5e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.600590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1235/2456] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:15:10.606443 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:10.623682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde17a-a52d-43cd-aefc-7d69ab6c4a3c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:10.628785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:10.629872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:10.630177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:10.630491 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] PATH_INFO: `/v3/projects/44f7c71ddce64c24acf7eea84534609b/users/2eed55ce903b45628fc28a9e747b235d/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:10.632560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f7c71ddce64c24acf7eea84534609b, user_id=2eed55ce903b45628fc28a9e747b235d, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:10.635288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:10.646814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1222/2457] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:15:10 2022] PUT /identity/v3/projects/44f7c71ddce64c24acf7eea84534609b/users/2eed55ce903b45628fc28a9e747b235d/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:15:15.135308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67ea-c365-4a7a-8c38-49cf18ab03b5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:15.135874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67ea-c365-4a7a-8c38-49cf18ab03b5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:15.136666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b67ea-c365-4a7a-8c38-49cf18ab03b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:15.169021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-1d4b67ea-c365-4a7a-8c38-49cf18ab03b5 None None] MFA Rules not processed for user `2eed55ce903b45628fc28a9e747b235d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:15.170503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1d4b67ea-c365-4a7a-8c38-49cf18ab03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:15.175565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:15.289778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1236/2458] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:15.310279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:15.312081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:15.333545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:15.407015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2ab80-e21b-4eef-a463-fe3f7e684338 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:15.415202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:15.416306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:15.416915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:15.417501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:15.429580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97725c6-de1b-40e9-af19-f85980664c64 None None] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:15.429953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97725c6-de1b-40e9-af19-f85980664c64 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:15.430402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97725c6-de1b-40e9-af19-f85980664c64 None None] PATH_INFO: `/` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:15.434866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1237/2459] 10.222.0.49 () {54 vars in 921 bytes} [Thu Jun 30 13:15:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 13:15:15.452313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:15.452602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:15.452879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:15.487144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] MFA Rules not processed for user `2a0b0da6c54341ff8fb66f48a0a32b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:15.488673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:15.498786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1238/2460] 10.222.0.49 () {58 vars in 1018 bytes} [Thu Jun 30 13:15:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:15.499428 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:15.501105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1abd2-23c6-4469-a0bd-e1e34ee43e43 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:15.510350 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:15.512440 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:15.521517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1223/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:15.610011 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c4b9-361d-438b-bf71-f71265acf2b7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:15.613817 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:15.614667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:15.614956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:15.615230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:15.626508 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:15.628040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:15.640027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1239/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:18.044355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445693-1dba-436c-a933-82a75c2fe3a5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:18.044858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445693-1dba-436c-a933-82a75c2fe3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:18.044953 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51445693-1dba-436c-a933-82a75c2fe3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:18.064970 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-51445693-1dba-436c-a933-82a75c2fe3a5 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:18.066202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-51445693-1dba-436c-a933-82a75c2fe3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:18.133484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1224/2463] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:15:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:18.145997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:18.148983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:18.227805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512b2c2-fcde-47a1-b2ef-84baacd40300 service glance] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:18.233386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:18.234047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:18.234347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:18.234622 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:18.243397 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:18.245052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:18.253326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1240/2464] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:15:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:15:28.125258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=74 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:28.249933 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=88 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:43.026201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:43.026802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:43.026802 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:43.046235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] MFA Rules not processed for user `1c4aa9e4d27c4c2ea4b49ab188f841da`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:43.047533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:43.055565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1225/2465] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:15:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:43.116589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:43.118445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:43.179591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a128323-3add-43b4-87a1-db58abc2f139 service swift] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:43.183620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:43.184269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:43.184552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:43.184826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:43.186702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:43.200740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:43.202173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1971e5-bc02-49bf-a8bb-4eacbb997cef service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:43.214038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1241/2466] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:43.616260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:43.618609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:43.677528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e7e811-a147-47b2-badf-afaf89a6ab6a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:43.681938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:43.682564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:43.682841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:43.683145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:43.685054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:43.699732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:43.701172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:43.712637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1226/2467] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:15:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:15:46.695799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:46.696307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:46.696603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:46.715607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:15:46.717121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:15:46.725272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1242/2468] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:15:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:15:46.739914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:15:46.754680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ba29-b7a6-45a7-93ad-547b4d049def service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:15:46.761909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:15:46.763018 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:15:46.763610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:15:46.764070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:15:46.778034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:15:46.779998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:15:46.791531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1227/2469] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:15:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:15:56.719801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=29 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:15:56.786720 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=35 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:03.098911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:03.108610 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d27944-5aa5-4f41-ad6c-46a61e2c8033 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:03.112862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:03.113510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:03.113797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:03.114067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:03.124701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:03.126135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:03.135644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1243/2470] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:12.791471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:12.801578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825eb88-a90c-419e-820f-521f77483253 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:12.805208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:12.805845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:12.806130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:12.806437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:12.808200 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:12.879750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:12.881214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:12.891794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1228/2471] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:13.131502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:16.720147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:16.745024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e638d4-7867-4bbe-a292-f19981257f27 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:16.762222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:16.763164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:16.763604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:16.764091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:16.766417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:16.790450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:16.791907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:16.807129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1244/2472] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:17.077512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.079082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:17.091060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0695e07-5054-49c6-85e6-700257ec9764 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.097139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.098026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.098281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.098760 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] PATH_INFO: `/v3/users/e0df4b789a804a00a267bcc5cf778324` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.114581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0df4b789a804a00a267bcc5cf778324)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.116922 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.167263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0df4b789a804a00a267bcc5cf778324'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.177195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0df4b789a804a00a267bcc5cf778324'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.177885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0df4b789a804a00a267bcc5cf778324'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.190598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0df4b789a804a00a267bcc5cf778324'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.191260 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-50a27962-c782-416f-9e7c-a9003fa5990b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0df4b789a804a00a267bcc5cf778324'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.203818 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1229/2473] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:17 2022] DELETE /identity/v3/users/e0df4b789a804a00a267bcc5cf778324 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:17.382298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.394444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4bd83-4abf-4dbc-93c8-124f762d7bd5 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.459298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.460168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.460463 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.460717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] PATH_INFO: `/v3/projects/2671e576de2c44e6b1f5df7c53aae76a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.472618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2671e576de2c44e6b1f5df7c53aae76a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.474718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.526577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2671e576de2c44e6b1f5df7c53aae76a', 'request_id': 'req-648eb9df-c794-445b-9c4b-43683428f4be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.536066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2671e576de2c44e6b1f5df7c53aae76a', 'request_id': 'req-648eb9df-c794-445b-9c4b-43683428f4be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.544361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2671e576de2c44e6b1f5df7c53aae76a', 'request_id': 'req-648eb9df-c794-445b-9c4b-43683428f4be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.557560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] The token cache is being invalidate because project 2671e576de2c44e6b1f5df7c53aae76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:17.557773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.558478 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-648eb9df-c794-445b-9c4b-43683428f4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:17.560096 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1245/2474] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:17 2022] DELETE /identity/v3/projects/2671e576de2c44e6b1f5df7c53aae76a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:17.565244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.565556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.565831 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.584803 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:17.586051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:17.638103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:17.654008 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1230/2475] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:17.662238 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.662578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.662877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.680556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:17.681789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:17.690066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1246/2476] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:17.694330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.704009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b4b02-4be2-4faa-946d-e4e7f686a48b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.708528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.709304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.709587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.709863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.710618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:16:17.710924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.712538 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.725054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1231/2477] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:16:17.729438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.739110 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99745b17-65d6-4165-a5cd-4d3ebee63a70 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.744653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.745308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.745588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.745881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.746589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.748546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.776488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1247/2478] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:17.780729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.791297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7058d4-88c7-49e8-85f1-beb9a49b9b1b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.849914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.850568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.850844 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.851106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.851827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.853765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.879533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1232/2479] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:17 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:17.880758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.891200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718054fb-420c-4737-bec4-b43a68e9b51f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.896111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.896770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.897172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.897447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.898176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.899819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.917958 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1248/2480] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:16:17.922301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.932776 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c4ac0-a156-432b-bdff-2cb4f6511bda admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.937259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.937939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.938219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.938534 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.939275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.940924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad0390-3244-41e5-8330-4b03982d47c9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:17.952211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1233/2481] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:17.956800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:17.967180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c978a6-8875-463c-bbfd-29d489eea0b1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:17.974766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:17.975993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:17.976338 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:17.976682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] PATH_INFO: `/v3/projects/cfe37fd672964ff88f4f362d9bf39759/users/697da1dfdd6946019996ceac10e76bea/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:17.993648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe37fd672964ff88f4f362d9bf39759, user_id=697da1dfdd6946019996ceac10e76bea, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:17.996502 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.013287 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1249/2482] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:17 2022] PUT /identity/v3/projects/cfe37fd672964ff88f4f362d9bf39759/users/697da1dfdd6946019996ceac10e76bea/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:18.023948 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.024481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.024824 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.056937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] MFA Rules not processed for user `697da1dfdd6946019996ceac10e76bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:18.058455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:18.127434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1234/2483] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:18.131668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.137358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:18.197368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea8ffab-3109-4e95-85e1-9af9e8abad16 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.203511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.203511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.203511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.203916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.204655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.206652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.241684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1250/2484] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:18 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:18.242996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.254604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb7174-ba35-4e98-a775-a14303c9fd7a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.309421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:18.321841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.322611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.322891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.323214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.323908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.325832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.351919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1235/2485] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:18 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:18.353225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.364916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35dc8f-c960-4fac-a7ac-06980d32ea5b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.369748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.370455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.370732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.371003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.371740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.373400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.388412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1251/2486] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:18.389539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.400245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d3351-aa63-4124-aa81-de929a779456 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.405211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.406176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.406499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.406771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] PATH_INFO: `/v3/projects/3a589025cb054ea5ad4fb192d6a87470/users/03b116ccd3134c1b83bac1f429045675/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.423307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a589025cb054ea5ad4fb192d6a87470, user_id=03b116ccd3134c1b83bac1f429045675, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.426053 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.441915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1236/2487] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:18 2022] PUT /identity/v3/projects/3a589025cb054ea5ad4fb192d6a87470/users/03b116ccd3134c1b83bac1f429045675/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:18.446690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.457722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f9cd2-2b4f-4dfc-b3ef-879a3ecf534e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.525251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.526102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.526652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.527051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.529524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.532192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc059c81-183a-43bd-8a78-91426f275ecd admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.548777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1252/2488] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:18.552105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.563481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b46cc-854f-4485-a6c9-3a39b6b4b538 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.568496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.569394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.569941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.570396 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] PATH_INFO: `/v3/projects/3a589025cb054ea5ad4fb192d6a87470/users/03b116ccd3134c1b83bac1f429045675/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.572317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a589025cb054ea5ad4fb192d6a87470, user_id=03b116ccd3134c1b83bac1f429045675, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.575021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.586858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1237/2489] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:18 2022] PUT /identity/v3/projects/3a589025cb054ea5ad4fb192d6a87470/users/03b116ccd3134c1b83bac1f429045675/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:18.599772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.600155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.600528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.629816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] MFA Rules not processed for user `03b116ccd3134c1b83bac1f429045675`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:18.631115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:18.701101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:18.715380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1253/2490] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:18.721040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.723503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:18.795636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40005d61-779c-4c75-8eb9-7e2182cf65ab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.800515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.801182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.801463 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.801744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.812263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.813820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58499e16-3909-4d00-b69a-f230b3a78f53 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.827270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1238/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:18.857135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:18.926628 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a67e0-ffdb-4c44-aee5-5c696a70fd73 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:18.931269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:18.932202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:18.932495 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:18.932754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] PATH_INFO: `/v3/users/697da1dfdd6946019996ceac10e76bea` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:18.934031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=697da1dfdd6946019996ceac10e76bea)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:18.936093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:18.998543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697da1dfdd6946019996ceac10e76bea'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.012980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697da1dfdd6946019996ceac10e76bea'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.013832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697da1dfdd6946019996ceac10e76bea'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.029398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697da1dfdd6946019996ceac10e76bea'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.030119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697da1dfdd6946019996ceac10e76bea'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.044256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1254/2492] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:18 2022] DELETE /identity/v3/users/697da1dfdd6946019996ceac10e76bea => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:19.048719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:19.049071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:19.049552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:19.067408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:19.068790 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:19.135580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1239/2493] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:19.142731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:19.144328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:19.208703 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf238d-1dc1-4f3d-9147-71bc43860604 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:19.213061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:19.213717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:19.214004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:19.214280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:19.231532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:19.233786 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:19.245097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1255/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:19.552119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:19.553738 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:19.568397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95c872-0ad7-4d69-8b8e-c7980fa503b6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:19.574161 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:19.575130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:19.576064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:19.576402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] PATH_INFO: `/v3/projects/cfe37fd672964ff88f4f362d9bf39759` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:19.577977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe37fd672964ff88f4f362d9bf39759)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:19.580251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:19.630749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:19.653605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe37fd672964ff88f4f362d9bf39759', 'request_id': 'req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.663900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe37fd672964ff88f4f362d9bf39759', 'request_id': 'req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.671633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe37fd672964ff88f4f362d9bf39759', 'request_id': 'req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.688409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] The token cache is being invalidate because project cfe37fd672964ff88f4f362d9bf39759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:19.688771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.690036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.692618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1240/2495] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:19 2022] DELETE /identity/v3/projects/cfe37fd672964ff88f4f362d9bf39759 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:19.696857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:19.709400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca113c9-10dd-4280-aef1-c5508c04f964 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:19.733250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:19.798310 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:19.799593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:19.799969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:19.800288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] PATH_INFO: `/v3/users/03b116ccd3134c1b83bac1f429045675` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:19.801514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b116ccd3134c1b83bac1f429045675)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:19.803498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:19.876886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b116ccd3134c1b83bac1f429045675'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.890932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b116ccd3134c1b83bac1f429045675'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.891789 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b116ccd3134c1b83bac1f429045675'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.906294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b116ccd3134c1b83bac1f429045675'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.907312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-903f63ce-7ffb-424c-9a9c-25dd05d3cb62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b116ccd3134c1b83bac1f429045675'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:19.925270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1256/2496] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:19 2022] DELETE /identity/v3/users/03b116ccd3134c1b83bac1f429045675 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:20.280644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.287113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:20.304212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92f3eb-5ca8-44e7-8125-88e8f5b3dd12 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.386917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.387878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.388128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.388439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] PATH_INFO: `/v3/projects/3a589025cb054ea5ad4fb192d6a87470` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.390183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a589025cb054ea5ad4fb192d6a87470)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.393618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.480589 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a589025cb054ea5ad4fb192d6a87470', 'request_id': 'req-75898e93-5b8a-41de-b328-21026241d999'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:20.495873 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a589025cb054ea5ad4fb192d6a87470', 'request_id': 'req-75898e93-5b8a-41de-b328-21026241d999'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:20.503029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a589025cb054ea5ad4fb192d6a87470', 'request_id': 'req-75898e93-5b8a-41de-b328-21026241d999'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:20.514397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] The token cache is being invalidate because project 3a589025cb054ea5ad4fb192d6a87470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:20.514688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:20.515725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-75898e93-5b8a-41de-b328-21026241d999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:20.517311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1241/2497] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:20 2022] DELETE /identity/v3/projects/3a589025cb054ea5ad4fb192d6a87470 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:20.522790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.523078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.523354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.551469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:20.552811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:20.634586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1257/2498] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:16:20.639370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.639660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.639940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.660646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:20.663042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:20.673804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1242/2499] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:16:20.678658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.694292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ceb7c-93f6-48f7-996a-1f6fcdc0c000 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.700804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.701563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.702046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.702428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.703464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:16:20.703880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.705625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.718566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1258/2500] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:16:20.723705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.733564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4838588-a482-42db-b8d2-aece82d26bb6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.738606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.739272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.739556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.739834 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.740558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.742596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.774833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1243/2501] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:20 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:16:20.776009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.786646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400d322-8c72-47ff-a094-16a2af07ba99 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.813920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:20.849224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.849938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.850226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.850558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.851272 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.853243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.875068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1259/2502] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:20 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:20.879457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.891386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afb0cd-5bce-46e5-bbef-c38338a045d3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.896184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.896944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.897298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.897641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.898479 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.901218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.913275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1244/2503] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:16:20.917499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.928846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bffcf-602d-4617-82d2-d523c451a8b0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.934313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.935109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.935487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.935887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.936924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.939153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a42524-4848-4dd6-a26c-ed2cf2b72f55 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.951247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1260/2504] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:20.955582 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:20.965791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5aced4-6400-41cb-a52d-749c59151927 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:20.972748 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:20.973594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:20.973886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:20.974132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] PATH_INFO: `/v3/projects/85468d939f3e47b39b5359c73bca92ad/users/e4fb43c33e22440bab13021660f1c9bc/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:20.990508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=85468d939f3e47b39b5359c73bca92ad, user_id=e4fb43c33e22440bab13021660f1c9bc, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:20.993231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:20.996814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:21.010357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1245/2505] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:20 2022] PUT /identity/v3/projects/85468d939f3e47b39b5359c73bca92ad/users/e4fb43c33e22440bab13021660f1c9bc/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:16:21.020656 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.020944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.021231 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.048404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] MFA Rules not processed for user `e4fb43c33e22440bab13021660f1c9bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:21.049614 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:21.118596 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1261/2506] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:21.122994 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.183186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853ece-7fd8-4965-bd4e-57b301e9eeeb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.188660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.189325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.189592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.189980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.190703 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.192621 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.223304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1246/2507] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:21 2022] POST /identity/v3/projects => generated 399 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:21.227807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.242493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05e0a8-210b-40e1-aa0d-95854646d320 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.308688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.309358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.309634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.309922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.310626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.312793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.337207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1262/2508] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:21 2022] POST /identity/v3/users => generated 306 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:21.341179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.352212 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404099a-e1fb-4c66-af9e-e3e9c9866443 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.356649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.357306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.357583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.357855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.358609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.360406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.372372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1247/2509] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:21.376664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.387344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d58654-88f8-4cea-9ec4-b3d81d60e989 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.392214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.393101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.393398 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.393663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] PATH_INFO: `/v3/projects/ebe06b09867b4923bb744519c2f21925/users/f1a2747485da4c238a865fcc8826e046/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.411112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe06b09867b4923bb744519c2f21925, user_id=f1a2747485da4c238a865fcc8826e046, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.413858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.431601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1263/2510] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:21 2022] PUT /identity/v3/projects/ebe06b09867b4923bb744519c2f21925/users/f1a2747485da4c238a865fcc8826e046/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:16:21.435788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.446950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88ca76-7b83-4111-a178-037b96da94ca admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.520706 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.521394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.521678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.521954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.522731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.524395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be12a3-d37a-49b3-b751-cd00f72dab37 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.535710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1248/2511] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:21.540904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.547895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:21.553590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9251dd2-22c6-4e92-b239-cf7e04177582 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.560024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.560740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.561127 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.561301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] PATH_INFO: `/v3/projects/ebe06b09867b4923bb744519c2f21925/users/f1a2747485da4c238a865fcc8826e046/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.563948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe06b09867b4923bb744519c2f21925, user_id=f1a2747485da4c238a865fcc8826e046, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.566666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.577444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1264/2512] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:21 2022] PUT /identity/v3/projects/ebe06b09867b4923bb744519c2f21925/users/f1a2747485da4c238a865fcc8826e046/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:21.588705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.588996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.589275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.616475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] MFA Rules not processed for user `f1a2747485da4c238a865fcc8826e046`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:21.617690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:21.669034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:21.689393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1249/2513] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:21 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:21.700575 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.703150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:21.760889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd043ab7-db95-4107-8c7e-246180487a60 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.764722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.765409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.765683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.765949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.774870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.776290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.786173 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1265/2514] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:21.864458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:21.875321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f19cf-b781-4fe6-bada-b3966d10011a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:21.879237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:21.879943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:21.880225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:21.880497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:21.954465 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:21.956291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:21.976439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1250/2515] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:23.461636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:23.463083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:23.536037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e21555-0ec5-4169-b5b2-a9b6cd3e3d8c service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:23.541177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:23.542064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:23.542370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:23.542645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] PATH_INFO: `/v3/users/9e51be11f09d44588e5b4ef657907fe6` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:23.543864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e51be11f09d44588e5b4ef657907fe6)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:23.545806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:23.599134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e51be11f09d44588e5b4ef657907fe6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.608645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e51be11f09d44588e5b4ef657907fe6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.609544 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e51be11f09d44588e5b4ef657907fe6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.627088 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e51be11f09d44588e5b4ef657907fe6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.628530 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e51be11f09d44588e5b4ef657907fe6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.641671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1266/2516] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:23 2022] DELETE /identity/v3/users/9e51be11f09d44588e5b4ef657907fe6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:23.826669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:23.838488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ca95f-651f-4ec0-964e-1cf6e43f8865 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:23.902269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:23.903182 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:23.903461 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:23.903732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] PATH_INFO: `/v3/projects/19004d450eee426786ca037d00f8fff4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:23.904937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19004d450eee426786ca037d00f8fff4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:23.907158 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:23.957174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19004d450eee426786ca037d00f8fff4', 'request_id': 'req-55acd52e-79d6-4069-a171-a223d5a4aed4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.965921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19004d450eee426786ca037d00f8fff4', 'request_id': 'req-55acd52e-79d6-4069-a171-a223d5a4aed4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.972410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19004d450eee426786ca037d00f8fff4', 'request_id': 'req-55acd52e-79d6-4069-a171-a223d5a4aed4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.983092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] The token cache is being invalidate because project 19004d450eee426786ca037d00f8fff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:23.983348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.984148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:23.985740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1251/2517] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:23 2022] DELETE /identity/v3/projects/19004d450eee426786ca037d00f8fff4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:23.989816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:23.991344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:24.004430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce16b7-6792-47ff-b40e-100d075c0da8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:24.012216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:24.063929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:24.064728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:24.065007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:24.065277 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] PATH_INFO: `/v3/users/9c9890d6755f4d77bfd2cb77694da9fb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:24.066519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9890d6755f4d77bfd2cb77694da9fb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:24.068455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:24.126555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9890d6755f4d77bfd2cb77694da9fb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.134345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9890d6755f4d77bfd2cb77694da9fb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.135037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9890d6755f4d77bfd2cb77694da9fb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.147499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9890d6755f4d77bfd2cb77694da9fb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.148181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9890d6755f4d77bfd2cb77694da9fb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.160671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1267/2518] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:23 2022] DELETE /identity/v3/users/9c9890d6755f4d77bfd2cb77694da9fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:16:24.347389 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:24.348848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:24.364054 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acd52e-79d6-4069-a171-a223d5a4aed4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:24.382555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:24.423520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:24.424384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:24.424927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:24.424927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] PATH_INFO: `/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:24.426116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35e17d7c66ac46629e5e0c5f2ad659e5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:24.428095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:24.477779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35e17d7c66ac46629e5e0c5f2ad659e5', 'request_id': 'req-8751e8b2-1374-4893-b529-a8a002f0e79c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.486604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35e17d7c66ac46629e5e0c5f2ad659e5', 'request_id': 'req-8751e8b2-1374-4893-b529-a8a002f0e79c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.493049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35e17d7c66ac46629e5e0c5f2ad659e5', 'request_id': 'req-8751e8b2-1374-4893-b529-a8a002f0e79c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.503773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] The token cache is being invalidate because project 35e17d7c66ac46629e5e0c5f2ad659e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:24.504058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.504707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8751e8b2-1374-4893-b529-a8a002f0e79c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:24.506284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1252/2519] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:24 2022] DELETE /identity/v3/projects/35e17d7c66ac46629e5e0c5f2ad659e5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:26.143840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:26.145260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:26.158782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613918f2-3295-4d09-b46c-66360eea2b0c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:26.221119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:26.222068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:26.222331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:26.222640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] PATH_INFO: `/v3/users/e4fb43c33e22440bab13021660f1c9bc` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:26.223956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fb43c33e22440bab13021660f1c9bc)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:26.225905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:26.267001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:26.282047 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fb43c33e22440bab13021660f1c9bc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.291191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fb43c33e22440bab13021660f1c9bc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.291806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fb43c33e22440bab13021660f1c9bc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.304265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fb43c33e22440bab13021660f1c9bc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.304963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fb43c33e22440bab13021660f1c9bc'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.317667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1268/2520] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:26 2022] DELETE /identity/v3/users/e4fb43c33e22440bab13021660f1c9bc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:16:26.322345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:26.322838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:26.322935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:26.341360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:26.342587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:26.402310 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1253/2521] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:16:26.412802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:26.414313 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:26.473232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f8aaa-f102-47ef-a1ef-090b13e4dcfe admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:26.477180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:26.477812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:26.478090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:26.478352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:26.487886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:26.489330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:26.499378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1269/2522] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:26.798627 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:26.800140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:26.814935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c1bc-f88d-49fb-a395-12255c9a7fda None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:26.820433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:26.821501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:26.821782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:26.822059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] PATH_INFO: `/v3/projects/85468d939f3e47b39b5359c73bca92ad` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:26.823353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85468d939f3e47b39b5359c73bca92ad)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:26.825306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:26.872630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:26.875254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85468d939f3e47b39b5359c73bca92ad', 'request_id': 'req-8e7fe946-2bbb-4d77-967b-c14ab4d54447'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.884359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85468d939f3e47b39b5359c73bca92ad', 'request_id': 'req-8e7fe946-2bbb-4d77-967b-c14ab4d54447'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.890625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85468d939f3e47b39b5359c73bca92ad', 'request_id': 'req-8e7fe946-2bbb-4d77-967b-c14ab4d54447'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.901273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] The token cache is being invalidate because project 85468d939f3e47b39b5359c73bca92ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:26.901556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.902174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:26.903765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1254/2523] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:26 2022] DELETE /identity/v3/projects/85468d939f3e47b39b5359c73bca92ad => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:26.907828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:26.918038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f4fe2-e80d-4152-a2b6-8bc41f503e74 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:26.978153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:26.979055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:26.979334 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:26.979626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] PATH_INFO: `/v3/users/f1a2747485da4c238a865fcc8826e046` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:26.980871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a2747485da4c238a865fcc8826e046)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:26.982847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.038300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a2747485da4c238a865fcc8826e046'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.047911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a2747485da4c238a865fcc8826e046'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.048649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a2747485da4c238a865fcc8826e046'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.050481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:27.063082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a2747485da4c238a865fcc8826e046'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.063748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a2df1d30-c2cd-46f8-ba3e-481ef5775a5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a2747485da4c238a865fcc8826e046'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.076388 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1270/2524] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:26 2022] DELETE /identity/v3/users/f1a2747485da4c238a865fcc8826e046 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:27.359417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.360881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:27.378245 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7fe946-2bbb-4d77-967b-c14ab4d54447 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.444977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.445825 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.446110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.446411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] PATH_INFO: `/v3/projects/ebe06b09867b4923bb744519c2f21925` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.447645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe06b09867b4923bb744519c2f21925)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.449604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.499019 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe06b09867b4923bb744519c2f21925', 'request_id': 'req-df2c9c67-db97-4d96-abb4-0eaf8c1de618'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.508737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe06b09867b4923bb744519c2f21925', 'request_id': 'req-df2c9c67-db97-4d96-abb4-0eaf8c1de618'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.515666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe06b09867b4923bb744519c2f21925', 'request_id': 'req-df2c9c67-db97-4d96-abb4-0eaf8c1de618'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.528084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] The token cache is being invalidate because project ebe06b09867b4923bb744519c2f21925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:27.528392 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.529022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-df2c9c67-db97-4d96-abb4-0eaf8c1de618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:27.530651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1255/2525] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:27 2022] DELETE /identity/v3/projects/ebe06b09867b4923bb744519c2f21925 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:27.536160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.536680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.536802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.556714 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:27.557856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:27.620344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1271/2526] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:16:27.629022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.629281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.629559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.650728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:27.651964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:27.661754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1256/2527] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:27.666270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.676783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a52f7-9372-40cc-83a3-dd3739574da4 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.682447 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.683108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.683393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.683677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.684419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:16:27.684723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.686343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.698631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1272/2528] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:16:27.703068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.713441 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0092bb-9ff6-427e-8768-c8723c3157f3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.719417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.720069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.720344 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.720910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.721700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.723713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.752827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1257/2529] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:27 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:27.757111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.767304 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6532bd-3721-40a2-b67f-a5d8cfd7fa08 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.828932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.829822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.830189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.830662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.831725 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.833729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.862202 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1273/2530] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:27 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:27.869132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.885116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00bc096-82f5-4618-b1d1-ba52df222beb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.892231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.892904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.893275 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.893548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.894307 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.895960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.907464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1258/2531] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:27.911552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.922620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697bd8-4506-469c-9d87-04177c148b7a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.928626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.929284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.929540 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.929835 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.930578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.932256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f89ca-aabb-43d7-84dc-b3011cc0390d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.943001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1274/2532] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:27.947231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:27.959176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05783b6-1f2e-4f73-9557-1397b7a5e83b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:27.964964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:27.965865 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:27.966142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:27.966432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] PATH_INFO: `/v3/projects/806d1c5a998d4de3b9a2bc02e833ce17/users/a57d31872aed43ab91bb4c72c1fd57aa/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:27.984401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=806d1c5a998d4de3b9a2bc02e833ce17, user_id=a57d31872aed43ab91bb4c72c1fd57aa, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:27.987121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:27.998667 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:28.004043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1259/2533] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:27 2022] PUT /identity/v3/projects/806d1c5a998d4de3b9a2bc02e833ce17/users/a57d31872aed43ab91bb4c72c1fd57aa/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:16:28.015340 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.015627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.015905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.043848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] MFA Rules not processed for user `a57d31872aed43ab91bb4c72c1fd57aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:28.045035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:28.071432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:28.116085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1275/2534] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:28.128288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:28.130203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:28.206002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a8617-9e85-4e02-bc57-fcdfb6352b5d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:28.212678 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:28.213552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.213945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.214438 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.226828 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:28.228292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeee85b-790a-4f3c-83ff-e1bec3cb92f0 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:28.240594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1260/2535] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:28.260176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:28.326731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16787ec1-02af-4252-a16b-f4ed8ec07545 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:28.331429 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:28.332411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.332716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.332996 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] PATH_INFO: `/v3/users/a57d31872aed43ab91bb4c72c1fd57aa` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.334237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a57d31872aed43ab91bb4c72c1fd57aa)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:28.336203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:28.383913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a57d31872aed43ab91bb4c72c1fd57aa'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.391648 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a57d31872aed43ab91bb4c72c1fd57aa'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.392281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a57d31872aed43ab91bb4c72c1fd57aa'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.404577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a57d31872aed43ab91bb4c72c1fd57aa'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.405234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a57d31872aed43ab91bb4c72c1fd57aa'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.418199 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1276/2536] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:28 2022] DELETE /identity/v3/users/a57d31872aed43ab91bb4c72c1fd57aa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:28.422799 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.423299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.423399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.441363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:28.442610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:28.515012 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1261/2537] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:28.525964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:28.527666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:28.581947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:28.587853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadc23f-6974-48f0-94de-b487aef9481c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:28.592738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:28.593607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.594102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.594609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.605707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:28.607182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30f358-2f20-448a-a886-6c32ca1c78be service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:28.617858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1277/2538] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:28.897575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:28.899072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:28.912016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971a8e-d7fb-468a-95ef-808b9f53cbfe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:28.918407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:28.919283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:28.919606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:28.919839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] PATH_INFO: `/v3/projects/806d1c5a998d4de3b9a2bc02e833ce17` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:28.921236 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=806d1c5a998d4de3b9a2bc02e833ce17)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:28.923073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:28.976301 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '806d1c5a998d4de3b9a2bc02e833ce17', 'request_id': 'req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.985052 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '806d1c5a998d4de3b9a2bc02e833ce17', 'request_id': 'req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:28.991903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '806d1c5a998d4de3b9a2bc02e833ce17', 'request_id': 'req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:29.003196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] The token cache is being invalidate because project 806d1c5a998d4de3b9a2bc02e833ce17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:29.003477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:29.004189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9d65aa90-0fd3-48a1-bf7a-ed8a2a3a56ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:29.005861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1262/2539] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:28 2022] DELETE /identity/v3/projects/806d1c5a998d4de3b9a2bc02e833ce17 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:29.011122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.011391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.011679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.029745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:29.031074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:29.097168 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1278/2540] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:29.106365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.106700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.106974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.131450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:29.133160 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:29.144864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1263/2541] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:29.149537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.159675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaf792-fe83-41ea-9f70-c8e1293aa50f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.165242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.166017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.166413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.166692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.167444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:16:29.167750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.169356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.182037 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1279/2542] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:16:29.186094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.196302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace954a6-ff09-4847-8fa8-44315a2fd4f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.202215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.202907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.203186 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.203455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.204126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.206114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.227864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:29.236943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1264/2543] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:29.241172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.251292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68de14d-5550-46ab-a6b2-0311719e7ccb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.319578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.320306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.320617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.320885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.321593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.323557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.347668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1280/2544] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:16:29.352737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.366836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d5b51-6759-4d63-8dbc-1da82d04bf2a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.372715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.373414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.373682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.373949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.374721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.376332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.387417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1265/2545] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:29.391505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.403991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72072550-e6e7-4ef4-9a84-018520eeb05e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.411642 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.412643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.413056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.413455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.414579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.416219 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741b74-7cfb-4d6b-a7df-f39e89fb1d16 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.429785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1281/2546] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:29.434148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.445498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32977671-51a7-4e56-86d7-0a624c380021 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.451264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.452184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.452469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.452777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] PATH_INFO: `/v3/projects/efa117c84c304bf3b59e03df78073546/users/9759270b749745c9b58a8f9023c46dc5/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.468599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efa117c84c304bf3b59e03df78073546, user_id=9759270b749745c9b58a8f9023c46dc5, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.471754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.487644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1266/2547] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:29 2022] PUT /identity/v3/projects/efa117c84c304bf3b59e03df78073546/users/9759270b749745c9b58a8f9023c46dc5/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:29.498368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.498658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.498931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.526907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] MFA Rules not processed for user `9759270b749745c9b58a8f9023c46dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:29.528804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:29.575805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:29.599067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1282/2548] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:29.603658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.672645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f9048-7265-4441-a58e-9e300f605651 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.678682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.679339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.679620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.679891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.680578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.682523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.712177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1267/2549] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:29.717717 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.729225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e23ce-4a16-4e0c-9c08-97db908c02e9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.801902 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.802782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.803177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.803576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.804586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.807222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.839118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1283/2550] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:29 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:29.840259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.851490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33001b-1a97-4035-ab76-a5a7f060abc1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.858022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.859004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.859442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.859872 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.860804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.862870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.874794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1268/2551] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:29.881706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.896556 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24cecf-41c5-4bc7-9586-4d9f71a7ef7f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:29.902803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:29.903692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:29.903971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:29.904247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] PATH_INFO: `/v3/projects/88c04b62a9cc4deb9518782661ab6b4a/users/0cc1e4d7f515470fab85fca1109aa01a/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:29.920134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c04b62a9cc4deb9518782661ab6b4a, user_id=0cc1e4d7f515470fab85fca1109aa01a, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:29.922929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:29.943178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1284/2552] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:29 2022] PUT /identity/v3/projects/88c04b62a9cc4deb9518782661ab6b4a/users/0cc1e4d7f515470fab85fca1109aa01a/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:29.949213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:29.961773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ff6-db17-4034-947d-ec4127a41568 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:30.022007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:30.022722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:30.023004 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:30.023277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:30.024009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:30.025618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ea926-7f55-4728-b5e4-a60428fedd53 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:30.039194 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1269/2553] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:30.040461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:30.051512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff45-5533-4325-8375-cfb24f8b19cb admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:30.056057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:30.056935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:30.057213 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:30.057481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] PATH_INFO: `/v3/projects/88c04b62a9cc4deb9518782661ab6b4a/users/0cc1e4d7f515470fab85fca1109aa01a/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:30.059453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c04b62a9cc4deb9518782661ab6b4a, user_id=0cc1e4d7f515470fab85fca1109aa01a, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:30.062083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:30.073243 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1285/2554] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:30 2022] PUT /identity/v3/projects/88c04b62a9cc4deb9518782661ab6b4a/users/0cc1e4d7f515470fab85fca1109aa01a/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:30.085468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:30.085757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:30.086040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:30.113171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] MFA Rules not processed for user `0cc1e4d7f515470fab85fca1109aa01a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:30.114417 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:30.137450 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:30.194804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1270/2555] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:30.206346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:30.208224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:30.279264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc01688-2189-4d66-8893-31e16c15c7ef admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:30.282912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:30.283558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:30.283841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:30.284112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:30.334611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:30.345147 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:30.347149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e0527-d46a-40a2-8e64-b8c0926ffc3b service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:30.356067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1286/2556] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:32.925202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:32.991449 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce044a-7e12-490f-9ee0-31e55dcf653b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:32.996602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:32.997478 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:32.997758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:32.998031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] PATH_INFO: `/v3/users/9759270b749745c9b58a8f9023c46dc5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:32.999291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9759270b749745c9b58a8f9023c46dc5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:33.001238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:33.053649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9759270b749745c9b58a8f9023c46dc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.062434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9759270b749745c9b58a8f9023c46dc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.063070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9759270b749745c9b58a8f9023c46dc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.075049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9759270b749745c9b58a8f9023c46dc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.075692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9759270b749745c9b58a8f9023c46dc5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.089395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1271/2557] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:32 2022] DELETE /identity/v3/users/9759270b749745c9b58a8f9023c46dc5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:33.094033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:33.094570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:33.094676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:33.112773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:33.114083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:33.174968 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1287/2558] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:16:33.185794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:33.188249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:33.239447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:33.264089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0486f0f-2dc8-4490-822b-def55c97b2ac admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:33.267933 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:33.268588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:33.268867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:33.269138 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:33.278788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:33.280240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:33.290082 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1272/2559] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:33.571215 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:33.572686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:33.585393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27c690-722c-439b-b3bf-42ba6b680963 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:33.590932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:33.591908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:33.592193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:33.592464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] PATH_INFO: `/v3/projects/efa117c84c304bf3b59e03df78073546` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:33.593765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efa117c84c304bf3b59e03df78073546)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:33.596240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:33.652875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efa117c84c304bf3b59e03df78073546', 'request_id': 'req-400ac86a-3656-4a4b-920d-26254d08f9a1'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.662851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efa117c84c304bf3b59e03df78073546', 'request_id': 'req-400ac86a-3656-4a4b-920d-26254d08f9a1'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.669552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efa117c84c304bf3b59e03df78073546', 'request_id': 'req-400ac86a-3656-4a4b-920d-26254d08f9a1'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.680268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] The token cache is being invalidate because project efa117c84c304bf3b59e03df78073546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:33.680551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.681232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.682915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1288/2560] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:33 2022] DELETE /identity/v3/projects/efa117c84c304bf3b59e03df78073546 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:33.687164 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:33.696949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a301ac3-6e53-45cd-a71b-31877c6a7b79 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:33.755073 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:33.755983 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:33.756244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:33.756512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] PATH_INFO: `/v3/users/0cc1e4d7f515470fab85fca1109aa01a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:33.757704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc1e4d7f515470fab85fca1109aa01a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:33.759652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:33.809810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc1e4d7f515470fab85fca1109aa01a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.819754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc1e4d7f515470fab85fca1109aa01a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.820756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc1e4d7f515470fab85fca1109aa01a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.838555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc1e4d7f515470fab85fca1109aa01a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.839347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94c47097-d4b7-4ff7-a4cf-4b47c322c073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc1e4d7f515470fab85fca1109aa01a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:33.852446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1273/2561] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:33 2022] DELETE /identity/v3/users/0cc1e4d7f515470fab85fca1109aa01a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:34.134474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.135943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:34.149193 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ac86a-3656-4a4b-920d-26254d08f9a1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.209211 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.209992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.210269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.210565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] PATH_INFO: `/v3/projects/88c04b62a9cc4deb9518782661ab6b4a` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.211779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c04b62a9cc4deb9518782661ab6b4a)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.213693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.231265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:34.265649 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c04b62a9cc4deb9518782661ab6b4a', 'request_id': 'req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:34.277143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c04b62a9cc4deb9518782661ab6b4a', 'request_id': 'req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:34.283853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c04b62a9cc4deb9518782661ab6b4a', 'request_id': 'req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:34.296085 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] The token cache is being invalidate because project 88c04b62a9cc4deb9518782661ab6b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:16:34.296461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:34.297258 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-5d705ce8-f3a9-4a9d-9747-a491f4c47c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:16:34.299013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1289/2562] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:16:34 2022] DELETE /identity/v3/projects/88c04b62a9cc4deb9518782661ab6b4a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:34.305145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.305646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.305767 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.334269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:34.335413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:34.391949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:34.402632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1274/2563] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:34.410577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.410883 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.411159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.430986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:34.432183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:34.440355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1290/2564] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:34.444806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.454904 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9de37-daa1-4f2f-a717-4efc0a0d8643 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.460227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.460811 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.461086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.461382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.462110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:16:34.462445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.464070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.476136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1275/2565] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:16:34.480422 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.490123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865a6d1-86fd-4fdb-a8b0-516647fe1f43 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.495460 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.496111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.496389 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.496660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.497345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.499269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.528187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1291/2566] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:16:34 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:34.532214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.543214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f40cb6-bdeb-424e-bebf-9ac50a768c4b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.600821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.601487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.601763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.602038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.602764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.604715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.631876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1276/2567] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:16:34 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:16:34.636252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.647143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8de289-fcf2-45ef-8685-de75c963b726 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.651922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.652570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.652848 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.653126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.653884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.655506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.665913 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1292/2568] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:34.670603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.687330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133a539-330b-4885-9849-edba0841cf52 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.695903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.696569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.696866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.697140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.697871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.699528 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb3754-2c6e-4de0-ae76-7f6f418eb61b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.711043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1277/2569] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:16:34.715206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.726100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929aec6c-ba9c-4a51-a852-6d31d5c93f1a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.731792 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.732668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.732947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.733216 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] PATH_INFO: `/v3/projects/876c1fcce45b4f02a90e79bd5073fffd/users/7cbbb92f022f4b339671956d32877eeb/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.749409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=876c1fcce45b4f02a90e79bd5073fffd, user_id=7cbbb92f022f4b339671956d32877eeb, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.752167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.770108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1293/2570] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:16:34 2022] PUT /identity/v3/projects/876c1fcce45b4f02a90e79bd5073fffd/users/7cbbb92f022f4b339671956d32877eeb/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:16:34.781380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.781856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.782309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.812215 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] MFA Rules not processed for user `7cbbb92f022f4b339671956d32877eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:16:34.813406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:34.882240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1278/2571] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:16:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:16:34.893738 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:16:34.895917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:16:34.953623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0efa3-38fb-4429-a77d-b1452119b3b7 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:16:34.957572 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:16:34.958206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:16:34.959192 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:16:34.959514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:16:34.969167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:16:34.970956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450ebe5-6243-476c-8be6-1c99af0dc545 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:16:34.980962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1294/2572] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:16:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:16:44.878810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:44.976681 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:16:59.934108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.000816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce62034-3dc7-4bdc-8294-06515c42f262 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.006013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.006926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.007201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.007471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] PATH_INFO: `/v3/users/7cbbb92f022f4b339671956d32877eeb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.008702 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbbb92f022f4b339671956d32877eeb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.010695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.063007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbbb92f022f4b339671956d32877eeb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.072949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbbb92f022f4b339671956d32877eeb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.073859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbbb92f022f4b339671956d32877eeb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.086475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbbb92f022f4b339671956d32877eeb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.087112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbbb92f022f4b339671956d32877eeb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.099435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1279/2573] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:16:59 2022] DELETE /identity/v3/users/7cbbb92f022f4b339671956d32877eeb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:00.104207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.104768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.104768 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.123003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:00.124312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.184031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1295/2574] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:00.195605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.197270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.258482 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af853d93-c0aa-4a5f-ab8e-fc9c49c0fcc5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.262606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.263263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.263535 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.263838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.274358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.275842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.287269 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1280/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:00.569878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.571468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.585294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dafea-7fce-4364-9c14-814a7a456a2e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.599043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.600684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.600852 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.601123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] PATH_INFO: `/v3/projects/876c1fcce45b4f02a90e79bd5073fffd` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.602695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=876c1fcce45b4f02a90e79bd5073fffd)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.605317 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.656841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '876c1fcce45b4f02a90e79bd5073fffd', 'request_id': 'req-6fab3302-add5-4662-8033-c48c4cb1c4a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.663578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.665678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '876c1fcce45b4f02a90e79bd5073fffd', 'request_id': 'req-6fab3302-add5-4662-8033-c48c4cb1c4a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.672624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '876c1fcce45b4f02a90e79bd5073fffd', 'request_id': 'req-6fab3302-add5-4662-8033-c48c4cb1c4a3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.673113 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534546e3-4384-4242-bc2d-77a6b1bb6735 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.684154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] The token cache is being invalidate because project 876c1fcce45b4f02a90e79bd5073fffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:00.684434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.685071 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-6fab3302-add5-4662-8033-c48c4cb1c4a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:00.686647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1296/2576] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:00 2022] DELETE /identity/v3/projects/876c1fcce45b4f02a90e79bd5073fffd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:00.691442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e2dcd-7d25-4cfe-9e5c-65eac1dab85f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.691730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e2dcd-7d25-4cfe-9e5c-65eac1dab85f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.692009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e2dcd-7d25-4cfe-9e5c-65eac1dab85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.695642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:00.711543 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-839e2dcd-7d25-4cfe-9e5c-65eac1dab85f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:00.712783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-839e2dcd-7d25-4cfe-9e5c-65eac1dab85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.730415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.731068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.731346 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.731618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.733496 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.773410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1297/2577] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:00.781803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.782075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.782380 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.797089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.798612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.800767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:00.801988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:00.808523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1281/2578] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:00.810049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1298/2579] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:00.814123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.823402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fad675-0a25-4b23-bc90-dae22a6ae18c service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.828120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.828781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.829063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.829334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.830116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:00.830451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.832143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.843985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1282/2580] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:17:00.848798 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.858092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39639135-c405-4d4a-a99d-5fc889a743c3 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.862788 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.863692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.864083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.864358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.865089 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.867160 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:00.881711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:00.897442 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1299/2581] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:00 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:17:00.901639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:00.912405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae05650-14f2-4c96-be27-b594b336e1e4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:00.968062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:00.970512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:00.970796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:00.971071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:00.971765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:00.973996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:01.000007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1283/2582] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:00 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:01.004528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:01.015200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b24c4-bc08-4efd-b8c5-9e46d7542dd3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:01.020060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:01.020895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:01.021299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:01.021634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:01.022476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:01.024151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:01.035615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1300/2583] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:01.039912 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:01.050342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc4578-2482-47ad-93ed-4e851ed9559e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:01.055555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:01.056225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:01.056505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:01.056779 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:01.057509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:01.059167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e202dd-448f-413a-a677-27d107269cc7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:01.069839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1284/2584] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:01.074118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:01.084545 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c958d6-f81a-4dbc-ba93-4d92e8413cf2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:01.090103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:01.091003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:01.091288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:01.091552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] PATH_INFO: `/v3/projects/7ba23248b8884303bfa7e4ad62f0f15d/users/95dc71187caf47a19da20e3a087f5bf7/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:01.107268 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba23248b8884303bfa7e4ad62f0f15d, user_id=95dc71187caf47a19da20e3a087f5bf7, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:01.109948 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:01.125017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1301/2585] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:01 2022] PUT /identity/v3/projects/7ba23248b8884303bfa7e4ad62f0f15d/users/95dc71187caf47a19da20e3a087f5bf7/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:01.135936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:01.136250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:01.136526 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:01.162771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] MFA Rules not processed for user `95dc71187caf47a19da20e3a087f5bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:01.163899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:01.192520 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:01.232781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1285/2586] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:01.243928 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:01.245651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:01.310057 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0a3fc-440f-4e4c-af98-0cd7f42071e1 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:01.313684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:01.314354 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:01.314637 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:01.314905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:01.323580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:01.325013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:01.334187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1302/2587] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:03.794437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:03.796127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:03.867304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b3e77-ee55-4a52-af2e-20bbb18ca666 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:03.872856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:03.873886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:03.874167 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:03.874469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] PATH_INFO: `/v3/users/126e2b6fc7114ec3bdc1bb3a99fcf752` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:03.889401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126e2b6fc7114ec3bdc1bb3a99fcf752)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:03.891409 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:03.938483 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126e2b6fc7114ec3bdc1bb3a99fcf752'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:03.947218 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126e2b6fc7114ec3bdc1bb3a99fcf752'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:03.947809 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126e2b6fc7114ec3bdc1bb3a99fcf752'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:03.959691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126e2b6fc7114ec3bdc1bb3a99fcf752'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:03.960311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e161a3e0-1079-4535-b5eb-6b08a8f03831 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126e2b6fc7114ec3bdc1bb3a99fcf752'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:03.972718 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1286/2588] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:03 2022] DELETE /identity/v3/users/126e2b6fc7114ec3bdc1bb3a99fcf752 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:04.153549 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.165475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1848ba-13a8-410d-a73a-0b9a36e184f2 service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.228708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.229494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.229774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.230045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] PATH_INFO: `/v3/projects/fb73f41e81224744800e167e0eb69863` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.241228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb73f41e81224744800e167e0eb69863)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.243187 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.297970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb73f41e81224744800e167e0eb69863', 'request_id': 'req-ed4913b0-da1f-4301-9057-ee7c4374c757'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:04.314251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb73f41e81224744800e167e0eb69863', 'request_id': 'req-ed4913b0-da1f-4301-9057-ee7c4374c757'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:04.315906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:04.323383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb73f41e81224744800e167e0eb69863', 'request_id': 'req-ed4913b0-da1f-4301-9057-ee7c4374c757'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:04.333657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] The token cache is being invalidate because project fb73f41e81224744800e167e0eb69863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:04.333946 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:04.334641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ed4913b0-da1f-4301-9057-ee7c4374c757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:04.337358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1303/2589] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:04 2022] DELETE /identity/v3/projects/fb73f41e81224744800e167e0eb69863 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:04.341966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.342247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.342543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.361080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:04.362298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:04.396437 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:04.434091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1287/2590] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:04.435843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.436129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.436406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.455369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:04.456611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:04.465602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1304/2591] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:04.470456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.480899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdc217-f2d4-4623-9379-7622615856f3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.485729 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.486420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.486705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.486969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.487692 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:04.488062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.489826 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.501917 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1288/2592] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:17:04.507500 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.518158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f80921-ebb2-47bc-908c-74439463d73e None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.523888 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.524542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.524822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.525094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.526086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.528467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.557235 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1305/2593] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:04.562788 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.578658 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc3aaa-b1c3-49a6-8679-129b654134b6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.672265 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.673268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.673775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.674266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.676123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.678660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.716131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1289/2594] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:17:04.720773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.736343 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c76631-39fc-4b8f-81b8-e43e9ee79405 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.745351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.746046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.746359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.746624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.747424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.749135 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.770214 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1306/2595] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:04.771806 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.782885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236dfb5-922b-43aa-a4ad-485c29bbd467 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.787850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.788651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.789066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.789402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.790345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.792003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80bb79-5265-4032-89bd-c5bc6fbaef41 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.803136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1290/2596] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:04.807513 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:04.817990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fedadb-2eae-45fd-9aac-a7102c17c1bd admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:04.823035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:04.823931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.824204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.824482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] PATH_INFO: `/v3/projects/a86579199e244f59b6d8c86c4531d132/users/86049905c7214a49bdb4b4a4451e5809/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.840695 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86579199e244f59b6d8c86c4531d132, user_id=86049905c7214a49bdb4b4a4451e5809, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:04.843475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:04.862521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1307/2597] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:04 2022] PUT /identity/v3/projects/a86579199e244f59b6d8c86c4531d132/users/86049905c7214a49bdb4b4a4451e5809/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:04.873959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:04.874395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:04.874689 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:04.905263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] MFA Rules not processed for user `86049905c7214a49bdb4b4a4451e5809`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:04.906592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:04.980536 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1291/2598] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:04.985472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.051326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e8d5-5c0e-4e6d-a8aa-9ee2c151e92d admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.057068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.057737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.058072 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.058465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.059149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.061155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.088423 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1308/2599] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:04 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:05.092596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.104595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92106c7e-11dc-4a93-baf6-0e6598f677dc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.136089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:05.165447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.166129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.166457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.166733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.167429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.169358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.195336 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1292/2600] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:05 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:17:05.196627 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.207721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1522b5-52fa-465a-8aac-dcf2673aa00f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.216212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.216874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.217158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.217434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.218170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.219837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.231533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1309/2601] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:05.235600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.245649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a2059c-921f-44bf-9581-2c44ed334fdb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.250685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.251601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.251879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.252153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] PATH_INFO: `/v3/projects/c9b9dede3a164565990728da672e1c6d/users/73572f18ac7644b1ad0b169d795edca3/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.270769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b9dede3a164565990728da672e1c6d, user_id=73572f18ac7644b1ad0b169d795edca3, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.273447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.288986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1293/2602] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:05 2022] PUT /identity/v3/projects/c9b9dede3a164565990728da672e1c6d/users/73572f18ac7644b1ad0b169d795edca3/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:05.293863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.306689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417cc9be-03d5-4fb3-b9f6-b788508553a9 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.365029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:05.376774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.377763 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.378230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.378726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.380512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.382550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65824b-634e-4421-9e06-9741251a38e5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.394992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1310/2603] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:05.399384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.410201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35947ee-1265-4c8e-b095-fb7c2339a0a9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.415613 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.416539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.416821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.417094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] PATH_INFO: `/v3/projects/c9b9dede3a164565990728da672e1c6d/users/73572f18ac7644b1ad0b169d795edca3/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.419256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b9dede3a164565990728da672e1c6d, user_id=73572f18ac7644b1ad0b169d795edca3, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.422014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.433360 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1294/2604] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:05 2022] PUT /identity/v3/projects/c9b9dede3a164565990728da672e1c6d/users/73572f18ac7644b1ad0b169d795edca3/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:17:05.443931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.444250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.444531 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.474291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] MFA Rules not processed for user `73572f18ac7644b1ad0b169d795edca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:05.475547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:05.560433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1311/2605] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:05.572752 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.575151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:05.636238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40257376-3809-486e-9958-429ddce83ab2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.640227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.640863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.641140 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.641412 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.651895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.653330 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.663015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1295/2606] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:05.740525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:05.742042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:05.755327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbdd64-b3d8-4bf2-8138-2540d97436d0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:05.759217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:05.759886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:05.760150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:05.760425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:05.818044 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:05.819453 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96811c8b-c965-4690-8e89-7c5cea3ff491 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:05.828573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1312/2607] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:14.833888 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:14.835691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:14.901209 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3410b1c8-f37f-4df6-9238-10d73013850c service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:14.906108 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:14.907010 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:14.907289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:14.907547 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] PATH_INFO: `/v3/users/95dc71187caf47a19da20e3a087f5bf7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:14.908793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=95dc71187caf47a19da20e3a087f5bf7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:14.910766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:14.940808 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:14.960507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95dc71187caf47a19da20e3a087f5bf7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:14.968900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95dc71187caf47a19da20e3a087f5bf7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:14.969666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95dc71187caf47a19da20e3a087f5bf7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:14.981632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95dc71187caf47a19da20e3a087f5bf7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:14.982246 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95dc71187caf47a19da20e3a087f5bf7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:14.995331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1296/2608] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:14 2022] DELETE /identity/v3/users/95dc71187caf47a19da20e3a087f5bf7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:14.999924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89598885-1153-40d7-a038-d82b968dddba None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.000427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89598885-1153-40d7-a038-d82b968dddba None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.000546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89598885-1153-40d7-a038-d82b968dddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.018177 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-89598885-1153-40d7-a038-d82b968dddba None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:15.019436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-89598885-1153-40d7-a038-d82b968dddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.081557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1313/2609] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:15.094079 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.095780 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.100601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89598885-1153-40d7-a038-d82b968dddba None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.102207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-89598885-1153-40d7-a038-d82b968dddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.121462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:15.155041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0763154-7ef8-4a33-8921-cb37291320ae admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:15.158717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:15.159365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.159652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.159926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.160657 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89598885-1153-40d7-a038-d82b968dddba None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:15.164809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:15.165448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.165731 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.166003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.167860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.169401 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:15.170897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:15.180055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:15.181488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5a0195-612d-488f-a53e-5d811b2752c5 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:15.183663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1297/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:15.191877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1314/2611] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:15.484723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.487040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.506982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d83c62-c7a6-4955-bd42-4831fc1668b2 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:15.512756 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:15.514086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.514620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.515126 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] PATH_INFO: `/v3/projects/7ba23248b8884303bfa7e4ad62f0f15d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.516930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba23248b8884303bfa7e4ad62f0f15d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:15.520869 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:15.590636 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba23248b8884303bfa7e4ad62f0f15d', 'request_id': 'req-1ea6b571-3ee8-40be-869a-711c9206b266'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:15.601596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba23248b8884303bfa7e4ad62f0f15d', 'request_id': 'req-1ea6b571-3ee8-40be-869a-711c9206b266'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:15.608292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba23248b8884303bfa7e4ad62f0f15d', 'request_id': 'req-1ea6b571-3ee8-40be-869a-711c9206b266'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:15.621957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] The token cache is being invalidate because project 7ba23248b8884303bfa7e4ad62f0f15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:15.622254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:15.622946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1ea6b571-3ee8-40be-869a-711c9206b266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:15.624564 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1298/2612] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:15 2022] DELETE /identity/v3/projects/7ba23248b8884303bfa7e4ad62f0f15d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:15.629555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.629849 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.630126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.650180 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:15.651818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.720624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1315/2613] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:15.729235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.729603 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.729950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.752457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:15.753726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:15.767092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1299/2614] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:15.768270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.780791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ee734-e10c-45a4-9ab5-8dff472c2041 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:15.786309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:15.787083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.787504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.787879 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.789485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:15.789905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:15.792346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:15.808752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1316/2615] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:17:15.810078 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.823231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d787960-3b6c-4b1b-acb7-62421987d08a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:15.828533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:15.829300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:15.829858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:15.830381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:15.831893 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:15.834170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:15.878347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1300/2616] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:15 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:15.932808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:15.943579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764bd3a-647c-4698-91dc-ea0f3c33ee68 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:16.015891 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:16.016581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.016847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.017125 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.017836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:16.019843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:16.041944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1317/2617] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:15 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:16.046045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:16.051540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:16.056679 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24186af4-edd9-48e4-ba82-43c1654a858c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:16.062648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:16.063472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.063739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.064035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.064907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:16.066960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:16.078066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1301/2618] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:16.082097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:16.092346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773966-bf09-486e-8dee-caf4a9224756 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:16.097751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:16.098522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.098819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.099091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.100633 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:16.101536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf636d-0a37-40e9-886a-280ce7eeb582 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:16.114119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1318/2619] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:16.118617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:16.130183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670516-3392-4a26-aaa3-d27c942b4240 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:16.136625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:16.138817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.139313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.139754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] PATH_INFO: `/v3/projects/42f6dedff3c34a50adcb8debc9caa217/users/6bce96f396304127b80d7de093664811/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.165733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f6dedff3c34a50adcb8debc9caa217, user_id=6bce96f396304127b80d7de093664811, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:16.168819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:16.184365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1302/2620] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:16 2022] PUT /identity/v3/projects/42f6dedff3c34a50adcb8debc9caa217/users/6bce96f396304127b80d7de093664811/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:16.195973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.196414 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.196698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.233153 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] MFA Rules not processed for user `6bce96f396304127b80d7de093664811`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:16.234503 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:16.246866 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:16.306730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1319/2621] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:16.318554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:16.320987 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:16.398773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c80fff-0145-4485-997d-0880118fb8e0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:16.403394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:16.404069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:16.404432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:16.404757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:16.415446 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:16.416919 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:16.426954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1303/2622] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:18.523048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:18.524607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:18.610945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:18.613288 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:18.663201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4daa8d6-037c-461e-92cc-5761b186e121 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:18.668893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:18.669841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:18.670341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:18.670829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:18.672851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:18.687191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:18.689018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:18.696114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa0e88-d538-4079-af59-409ba9906633 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:18.701841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:18.705089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:18.705089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:18.705089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] PATH_INFO: `/v3/users/e5e99d5484594d13b113f27e77b37efd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:18.705435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1320/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:18.726665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e99d5484594d13b113f27e77b37efd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:18.728836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:18.778984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e99d5484594d13b113f27e77b37efd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:18.794791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e99d5484594d13b113f27e77b37efd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:18.795447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e99d5484594d13b113f27e77b37efd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:18.801907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:18.811130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e99d5484594d13b113f27e77b37efd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:18.812112 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d780ef3f-4bf8-4bb1-8a80-3fe93d84ee3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e99d5484594d13b113f27e77b37efd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:18.825234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1304/2624] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:18 2022] DELETE /identity/v3/users/e5e99d5484594d13b113f27e77b37efd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:17:19.016808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.028020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8994f29-c8c8-4486-b692-c4831087dd7b service placement] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.106967 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.107843 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.108120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.108390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] PATH_INFO: `/v3/projects/cee8ced67bdf4b5ca8aa12a8940b9bea` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.119723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee8ced67bdf4b5ca8aa12a8940b9bea)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.121635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.161542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:19.171431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee8ced67bdf4b5ca8aa12a8940b9bea', 'request_id': 'req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:19.180033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee8ced67bdf4b5ca8aa12a8940b9bea', 'request_id': 'req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:19.186949 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee8ced67bdf4b5ca8aa12a8940b9bea', 'request_id': 'req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:19.198008 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] The token cache is being invalidate because project cee8ced67bdf4b5ca8aa12a8940b9bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:19.198299 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:19.199031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2bde8c5b-8d6a-45ae-9fb5-12019c872a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:19.200670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1321/2625] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:19 2022] DELETE /identity/v3/projects/cee8ced67bdf4b5ca8aa12a8940b9bea => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:19.205812 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.206092 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.206374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.231793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:19.233615 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:19.301644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1305/2626] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:19.312294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.312585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.312930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.335029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:19.336353 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:19.346056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1322/2627] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:19.350827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.361434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a2f65-d216-48eb-8aea-b3cda09d1d56 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.367369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.368029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.368308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.368581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.369313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:19.369617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.371262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.384081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1306/2628] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:17:19.388451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.397932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105898e-ccf1-4ac7-be06-eb1a422fbed8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.404212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.404864 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.405145 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.405419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.406143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.408079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.437189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1323/2629] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:19.441375 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.451522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406de8e-a954-40ca-82be-14ceceb938ba admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.516045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.516653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.516915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.517208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.517924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.519885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.541751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1307/2630] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:17:19.545912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.556630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088da545-ccc9-4d66-99d0-fb1c4545357b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.561609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.562275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.562589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.562860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.563585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.565201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.576086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1324/2631] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:19.580786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.597096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4a330-2699-4d01-852e-eb561fe5f913 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.604104 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.604764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.605040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.605317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.606048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.607698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01398929-8513-4f3f-b386-79d702d6ee4c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.618386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1308/2632] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:19.622801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.633704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5623b537-40a7-4521-bcd7-ef7d4745ff50 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.639117 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.640003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.640286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.640554 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] PATH_INFO: `/v3/projects/a352272b75764c599096c42f39b1fc7e/users/bcf7829370f04027a1156a8c627081d5/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.655399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a352272b75764c599096c42f39b1fc7e, user_id=bcf7829370f04027a1156a8c627081d5, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.658077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.672479 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1325/2633] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:19 2022] PUT /identity/v3/projects/a352272b75764c599096c42f39b1fc7e/users/bcf7829370f04027a1156a8c627081d5/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:19.683758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.684044 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.684311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.713798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] MFA Rules not processed for user `bcf7829370f04027a1156a8c627081d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:19.715088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:19.749676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:19.785439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1309/2634] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:19.790351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.853700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f96026-d3b7-4c9c-a32a-13086dfa1116 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.858871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.859487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.859797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.860095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.860825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.862766 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.891488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1326/2635] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:19.895471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:19.907075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e92f1a-3d7e-4585-be00-a60af2619729 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:19.969622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:19.970661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:19.970940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:19.971216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:19.971907 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:19.973848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:19.998859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1310/2636] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:19 2022] POST /identity/v3/users => generated 308 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:17:20.002988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:20.013911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a093e21-4bcd-416b-9848-479d1fe89bf3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:20.019122 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:20.019781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.020058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.020327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.021060 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:20.022784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:20.033504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1327/2637] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:20.038225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:20.048555 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7765424-04de-41df-a13b-3172a145879e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:20.054286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:20.055195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.055471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.055740 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] PATH_INFO: `/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d/users/4c7d99cb014240ad9d952567870c2afb/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.071560 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eb9cb3a1b54709a84d0e8a50006c5d, user_id=4c7d99cb014240ad9d952567870c2afb, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:20.074192 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:20.091657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1311/2638] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:20 2022] PUT /identity/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d/users/4c7d99cb014240ad9d952567870c2afb/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:20.096083 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:20.106890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d26487-794d-47ab-acc0-f64eff848aae admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:20.139823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:20.166599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:20.167254 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.167533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.167808 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.168523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:20.170118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5496d3e5-95a2-47b6-936f-d96178c1dda1 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:20.180206 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1328/2639] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:20.184670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:20.195500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9889-737f-4083-a769-45527d944701 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:20.201279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:20.202172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.202484 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.202755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] PATH_INFO: `/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d/users/4c7d99cb014240ad9d952567870c2afb/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.204899 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=20eb9cb3a1b54709a84d0e8a50006c5d, user_id=4c7d99cb014240ad9d952567870c2afb, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:20.207571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:20.218353 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1312/2640] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:20 2022] PUT /identity/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d/users/4c7d99cb014240ad9d952567870c2afb/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:20.228586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.228874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.229149 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.256847 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] MFA Rules not processed for user `4c7d99cb014240ad9d952567870c2afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:20.258051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:20.329677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1329/2641] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:20.345290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:20.347297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:20.407847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1da45-5f6c-45e5-b19c-dd06299bf5fd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:20.412846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:20.413490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:20.413785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:20.414061 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:20.423408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:20.424851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f21782-a6d4-4b63-8e3b-b5f1bb46e700 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:20.434580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1313/2642] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:22.181410 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:22.268851 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af475fab-0110-45a2-a4c0-bbc2c360ec6a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:22.275068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:22.276520 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:22.276743 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:22.278927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] PATH_INFO: `/v3/users/bcf7829370f04027a1156a8c627081d5` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:22.278927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf7829370f04027a1156a8c627081d5)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:22.280359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:22.342361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf7829370f04027a1156a8c627081d5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.353844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf7829370f04027a1156a8c627081d5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.354870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf7829370f04027a1156a8c627081d5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.361887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:22.372609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf7829370f04027a1156a8c627081d5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.373811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf7829370f04027a1156a8c627081d5'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.392761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1330/2643] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:22 2022] DELETE /identity/v3/users/bcf7829370f04027a1156a8c627081d5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:22.398350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:22.398883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:22.398883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:22.418083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:22.419541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:22.451040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:22.497506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1314/2644] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:22.508733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:22.510400 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:22.583698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e799a75-f959-4258-995c-bb7821785070 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:22.590103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:22.591184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:22.591582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:22.591961 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:22.602904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:22.604396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:22.615009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1331/2645] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:22.890521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:22.892081 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:22.905690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaea493-5274-4bc9-a385-c03040238cfb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:22.911315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:22.912492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:22.912838 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:22.913176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] PATH_INFO: `/v3/projects/a352272b75764c599096c42f39b1fc7e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:22.914680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a352272b75764c599096c42f39b1fc7e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:22.916565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:22.973195 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a352272b75764c599096c42f39b1fc7e', 'request_id': 'req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.982259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a352272b75764c599096c42f39b1fc7e', 'request_id': 'req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:22.988815 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a352272b75764c599096c42f39b1fc7e', 'request_id': 'req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.002101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] The token cache is being invalidate because project a352272b75764c599096c42f39b1fc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:23.002101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.002914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.005090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1315/2646] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:22 2022] DELETE /identity/v3/projects/a352272b75764c599096c42f39b1fc7e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:23.009200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:23.019077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d08c3-d439-410f-9fd4-0f3f14693655 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:23.077925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:23.078932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.079220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.079584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] PATH_INFO: `/v3/users/4c7d99cb014240ad9d952567870c2afb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.080700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7d99cb014240ad9d952567870c2afb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:23.082690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:23.133813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7d99cb014240ad9d952567870c2afb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.142012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7d99cb014240ad9d952567870c2afb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.142778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7d99cb014240ad9d952567870c2afb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.155372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7d99cb014240ad9d952567870c2afb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.156035 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-292b449f-513d-49ca-9d68-b746697a6c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7d99cb014240ad9d952567870c2afb'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.164937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:23.174767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1332/2647] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:23 2022] DELETE /identity/v3/users/4c7d99cb014240ad9d952567870c2afb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:23.483433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:23.484966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:23.500682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca5d6-a5d8-4e31-9ca2-cc31bc17f059 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:23.564820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:23.566084 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.566362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.566663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] PATH_INFO: `/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.568633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20eb9cb3a1b54709a84d0e8a50006c5d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:23.570599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:23.624100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20eb9cb3a1b54709a84d0e8a50006c5d', 'request_id': 'req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.627902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:23.634704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20eb9cb3a1b54709a84d0e8a50006c5d', 'request_id': 'req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.642786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20eb9cb3a1b54709a84d0e8a50006c5d', 'request_id': 'req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.652933 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] The token cache is being invalidate because project 20eb9cb3a1b54709a84d0e8a50006c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:23.653220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.653847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4abeb1a6-f9df-4105-9c23-b3649a0dcf0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:23.655422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1316/2648] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:23 2022] DELETE /identity/v3/projects/20eb9cb3a1b54709a84d0e8a50006c5d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:23.661716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.662391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.662517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.681242 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:23.682487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:23.752690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1333/2649] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:23.760859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.761172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.761447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.780687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:23.781902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:23.791557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1317/2650] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:23.795956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:23.806608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ee393-1f78-44aa-9bb4-f51816707d6f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:23.812425 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:23.813101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.813377 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.813647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.814364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:23.814701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:23.816307 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:23.828403 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1334/2651] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:17:23.832750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:23.843293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697d8f9-8cd0-4ea1-bdc9-3e3650351438 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:23.849216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:23.850033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.850305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.850601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.851293 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:23.853204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:23.888761 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1318/2652] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:23 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:23.893911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:23.905956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc011c4-6224-434f-b7c5-70325ca462b5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:23.972607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:23.973288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:23.973571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:23.973846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:23.974585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:23.976515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:24.003431 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1335/2653] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:23 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:24.009214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:24.020250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897d5f2-115d-45a2-814a-2aea978c7b0d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:24.025403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:24.026059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.026341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.026654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.027383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:24.029016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:24.039898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1319/2654] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:24.043909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:24.053940 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769fa221-3caf-4ae3-8198-6042f1a1a6b9 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:24.058832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:24.059473 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.059753 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.060022 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.060750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:24.062361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7fef9-3e79-486d-a6a6-0f76ab1fb165 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:24.072589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1336/2655] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:24.076661 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:24.087680 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587d8f0-2389-41d0-80ca-b5f926ffb42a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:24.093136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:24.094013 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.094292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.094606 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] PATH_INFO: `/v3/projects/7673badca23a4aa58d9f361e8ff5044e/users/87d8bebdd96b4ef89857c016f66ad263/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.110704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7673badca23a4aa58d9f361e8ff5044e, user_id=87d8bebdd96b4ef89857c016f66ad263, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:24.113372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:24.132952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1320/2656] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:24 2022] PUT /identity/v3/projects/7673badca23a4aa58d9f361e8ff5044e/users/87d8bebdd96b4ef89857c016f66ad263/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:17:24.138115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a1990-bbdb-424b-a9ca-e65f90580f04 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.138433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a1990-bbdb-424b-a9ca-e65f90580f04 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.138708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a1990-bbdb-424b-a9ca-e65f90580f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.157515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-204a1990-bbdb-424b-a9ca-e65f90580f04 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:24.158758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-204a1990-bbdb-424b-a9ca-e65f90580f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:24.219155 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1337/2657] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:24.229370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:24.231183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:24.300519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f1b62-98cb-4c4f-9320-3a2d1fbdc670 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:24.307989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:24.308639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.308921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.309220 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.319127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:24.320628 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:24.330736 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1321/2658] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:24.984686 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:24.985255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:24.985255 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:24.993601 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:25.014876 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] MFA Rules not processed for user `87d8bebdd96b4ef89857c016f66ad263`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:25.016169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:25.084150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1338/2659] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:25.089315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.100106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ce4f9-7e09-429d-ae3b-0b8f52b19fac service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.104675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.105327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.105604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.105871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.106652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.108566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.140575 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1322/2660] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:25 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:25.142021 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.153241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e006be-b895-4d04-9d88-2164251f6cc8 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.208043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.208712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.209050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.209326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.210012 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.211980 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.234704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1339/2661] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:25 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:25.238785 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.249277 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28311424-a5a4-4af8-bd47-6885bbb61ff0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.253937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.254626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.254913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.255188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.255979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.257602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.269416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1323/2662] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:25.273839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.284029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476338f3-a998-4514-96ba-66d538bb103e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.289472 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.290359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.290654 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.290925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] PATH_INFO: `/v3/projects/f49594ae14d54616a219d61043f26e6f/users/ff8c34c4d0c94f3ebb09e5af75145016/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.307647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49594ae14d54616a219d61043f26e6f, user_id=ff8c34c4d0c94f3ebb09e5af75145016, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.311828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.330819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1340/2663] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:25 2022] PUT /identity/v3/projects/f49594ae14d54616a219d61043f26e6f/users/ff8c34c4d0c94f3ebb09e5af75145016/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:25.334908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.345216 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4ad58-4032-44a2-b021-70d23e02f02c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.390177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:25.403187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.403861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.404143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.404415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.405163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.406814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a6b46-c2b9-4a0b-8771-322559e1921f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.416685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1324/2664] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:17:25.420790 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:25.432335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85b126-bf2f-4426-94ad-44777b8c2e93 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:25.437673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:25.438602 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:25.438919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:25.439158 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] PATH_INFO: `/v3/projects/f49594ae14d54616a219d61043f26e6f/users/ff8c34c4d0c94f3ebb09e5af75145016/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:25.441222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49594ae14d54616a219d61043f26e6f, user_id=ff8c34c4d0c94f3ebb09e5af75145016, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:25.443908 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:25.454547 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1341/2665] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:25 2022] PUT /identity/v3/projects/f49594ae14d54616a219d61043f26e6f/users/ff8c34c4d0c94f3ebb09e5af75145016/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:17:26.100326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:26.100878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:26.100878 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:26.128915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] MFA Rules not processed for user `ff8c34c4d0c94f3ebb09e5af75145016`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:26.130190 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:26.207023 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1325/2666] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:26.218433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:26.220538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:26.277086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:26.282765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f86af-92cc-4c76-ba7c-d8de5f030fe8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:26.287328 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:26.288210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:26.288663 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:26.289002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:26.348697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:26.350452 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:26.359538 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1342/2667] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:29.832991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:29.834655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:29.847995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e67540-3bbb-4db4-b596-bd51598246f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:29.852055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:29.852879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:29.853356 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:29.853750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:29.863335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:29.864781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5715b0-6586-4a89-8d6d-b27f14389fc1 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:29.875015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1326/2668] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:29 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:36.355536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=27 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:39.018103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:39.019735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:39.088918 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78c81f-5118-4ccc-af55-90b5cff933ac service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:39.094970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:39.095836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:39.096112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:39.096376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] PATH_INFO: `/v3/users/6bce96f396304127b80d7de093664811` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:39.097673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bce96f396304127b80d7de093664811)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:39.099646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:39.152115 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bce96f396304127b80d7de093664811'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.161481 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bce96f396304127b80d7de093664811'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.162150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bce96f396304127b80d7de093664811'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.175009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bce96f396304127b80d7de093664811'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.175634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bce96f396304127b80d7de093664811'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.188305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1343/2669] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:39 2022] DELETE /identity/v3/users/6bce96f396304127b80d7de093664811 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:39.193000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:39.193515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:39.193616 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:39.212481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:39.213699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:39.283345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1327/2670] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:39.294826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:39.296616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:39.359062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2848c4d-26f8-44a3-ae2d-fcc483d91f71 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:39.363103 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:39.363746 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:39.364026 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:39.364305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:39.374051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:39.375857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:39.385834 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1344/2671] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:39.676804 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:39.679009 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:39.692047 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ff27b-fe98-47cf-a16c-83a9946813c0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:39.712676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:39.713652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:39.713903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:39.714249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] PATH_INFO: `/v3/projects/42f6dedff3c34a50adcb8debc9caa217` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:39.715633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f6dedff3c34a50adcb8debc9caa217)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:39.717568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:39.772586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f6dedff3c34a50adcb8debc9caa217', 'request_id': 'req-30848d33-f40b-4eb9-97d4-c431f6516fcb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.775965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:39.782294 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f6dedff3c34a50adcb8debc9caa217', 'request_id': 'req-30848d33-f40b-4eb9-97d4-c431f6516fcb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.789505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f6dedff3c34a50adcb8debc9caa217', 'request_id': 'req-30848d33-f40b-4eb9-97d4-c431f6516fcb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.801066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] The token cache is being invalidate because project 42f6dedff3c34a50adcb8debc9caa217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:39.801384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.802083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:39.803734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1328/2672] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:39 2022] DELETE /identity/v3/projects/42f6dedff3c34a50adcb8debc9caa217 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:49.381239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:49.798021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:52.515677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:52.517233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:52.531583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819fe8e-e5fc-4f9b-9a1d-4d4add15344f service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:52.593031 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:52.593917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:52.594200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:52.594853 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] PATH_INFO: `/v3/users/87d8bebdd96b4ef89857c016f66ad263` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:52.596128 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d8bebdd96b4ef89857c016f66ad263)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:52.598082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:52.648858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d8bebdd96b4ef89857c016f66ad263'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.656590 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d8bebdd96b4ef89857c016f66ad263'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.657208 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d8bebdd96b4ef89857c016f66ad263'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.668841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d8bebdd96b4ef89857c016f66ad263'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.669469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d8bebdd96b4ef89857c016f66ad263'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.681911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1345/2673] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:52 2022] DELETE /identity/v3/users/87d8bebdd96b4ef89857c016f66ad263 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:52.855339 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:52.857085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:52.870941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30848d33-f40b-4eb9-97d4-c431f6516fcb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:52.928238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:52.929022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:52.929291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:52.929563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] PATH_INFO: `/v3/projects/7673badca23a4aa58d9f361e8ff5044e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:52.930813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7673badca23a4aa58d9f361e8ff5044e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:52.932725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:52.979666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7673badca23a4aa58d9f361e8ff5044e', 'request_id': 'req-7d65cc90-3f40-45b0-91e8-bcb7667158d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.989254 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7673badca23a4aa58d9f361e8ff5044e', 'request_id': 'req-7d65cc90-3f40-45b0-91e8-bcb7667158d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:52.995894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7673badca23a4aa58d9f361e8ff5044e', 'request_id': 'req-7d65cc90-3f40-45b0-91e8-bcb7667158d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.006971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] The token cache is being invalidate because project 7673badca23a4aa58d9f361e8ff5044e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:53.007262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.007937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.009530 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1329/2674] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:52 2022] DELETE /identity/v3/projects/7673badca23a4aa58d9f361e8ff5044e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:53.013800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.015337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:53.029399 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc63392-8bd9-405d-9816-05922070f245 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.100284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.101358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.101666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.101924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] PATH_INFO: `/v3/users/ff8c34c4d0c94f3ebb09e5af75145016` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.103369 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8c34c4d0c94f3ebb09e5af75145016)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.105326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.127227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:53.155042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8c34c4d0c94f3ebb09e5af75145016'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.164382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8c34c4d0c94f3ebb09e5af75145016'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.165099 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8c34c4d0c94f3ebb09e5af75145016'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.176953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8c34c4d0c94f3ebb09e5af75145016'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.177578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-22aff39e-69a8-4a85-a83d-b761ded8edfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8c34c4d0c94f3ebb09e5af75145016'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.191411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1346/2675] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:17:53 2022] DELETE /identity/v3/users/ff8c34c4d0c94f3ebb09e5af75145016 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:53.381457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.383210 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:53.397837 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65cc90-3f40-45b0-91e8-bcb7667158d7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.458928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.460249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.460489 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.460789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] PATH_INFO: `/v3/projects/f49594ae14d54616a219d61043f26e6f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.462137 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f49594ae14d54616a219d61043f26e6f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.464162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.502393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:53.516253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f49594ae14d54616a219d61043f26e6f', 'request_id': 'req-932cd0ae-a1c1-4036-be62-69361d93e7dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.526007 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f49594ae14d54616a219d61043f26e6f', 'request_id': 'req-932cd0ae-a1c1-4036-be62-69361d93e7dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.534558 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f49594ae14d54616a219d61043f26e6f', 'request_id': 'req-932cd0ae-a1c1-4036-be62-69361d93e7dd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.547383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] The token cache is being invalidate because project f49594ae14d54616a219d61043f26e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:17:53.547383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.548029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-932cd0ae-a1c1-4036-be62-69361d93e7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:17:53.549591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1330/2676] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:17:53 2022] DELETE /identity/v3/projects/f49594ae14d54616a219d61043f26e6f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:53.556494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.557169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.557169 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.580587 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:53.581890 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:53.646303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1347/2677] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:53.655355 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.655646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.655921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.677085 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:53.678429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:53.687129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1331/2678] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:53.691947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.702787 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a34a7c-3e2a-4bdd-8b83-50c5a3c86a8d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.708455 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.709118 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.709395 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.709692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.710459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:17:53.710767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.712393 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.725297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1348/2679] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:17:53.729851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.739724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82374d97-e2e8-4c78-96c6-2807ee48abb7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.745882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.746573 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.746856 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.747152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.747841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.749772 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.785394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1332/2680] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:17:53 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:53.791230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.802578 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b7575-a5e0-4c89-9c7b-feabe3b8e0de admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.861330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.861983 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.862286 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.862592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.863281 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.865186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.890733 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1349/2681] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:17:53 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:17:53.892056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.902425 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c47470-858b-4656-a974-93cdfe67afbe admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.907132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.907784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.908062 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.908332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.909080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.910713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.920988 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1333/2682] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:53.924973 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.936097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096929be-742a-4f8f-a902-dc6e6df5daab admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.941931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.942605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.942887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.943172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.943893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.945828 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a38af-898b-4068-bd15-0292d4ca873a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:53.956509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1350/2683] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:17:53.961096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:53.972949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8b9f7-becd-468b-b03d-e92dbaec07e9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:53.977931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:53.978857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:53.979131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:53.979403 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] PATH_INFO: `/v3/projects/3f1179a608f64ede9eb3c4252619f823/users/e2f423922c784da2be2d16c736a1ec16/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:53.995034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1179a608f64ede9eb3c4252619f823, user_id=e2f423922c784da2be2d16c736a1ec16, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:53.997704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:54.014855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1334/2684] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:17:53 2022] PUT /identity/v3/projects/3f1179a608f64ede9eb3c4252619f823/users/e2f423922c784da2be2d16c736a1ec16/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:17:54.022280 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23060-8279-4e84-ad28-eecd0de14ade None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:54.022741 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23060-8279-4e84-ad28-eecd0de14ade None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:54.022997 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee23060-8279-4e84-ad28-eecd0de14ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:54.031496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:54.043842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-7ee23060-8279-4e84-ad28-eecd0de14ade None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:54.045734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-7ee23060-8279-4e84-ad28-eecd0de14ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:54.108723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1351/2685] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:54.122193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:54.125094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:54.194255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa732cba-69ca-4abd-8a90-7af6e9cff751 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:54.198481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:54.199123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:54.199400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:54.199670 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:54.209089 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:54.210498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:54.220413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1335/2686] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:54.866138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8faef-5df6-41fe-abf0-da09bb0b9057 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:54.866965 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8faef-5df6-41fe-abf0-da09bb0b9057 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:54.867434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8faef-5df6-41fe-abf0-da09bb0b9057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:54.894565 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-34f8faef-5df6-41fe-abf0-da09bb0b9057 None None] MFA Rules not processed for user `e2f423922c784da2be2d16c736a1ec16`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:54.895782 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-34f8faef-5df6-41fe-abf0-da09bb0b9057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:54.962165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1352/2687] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:17:54.973975 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:54.975610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:55.032435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719e9af-1aa0-4bf7-ba1f-e3eb76bafa7a service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:55.036110 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:55.036758 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:55.037057 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:55.037333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:55.045944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:55.047369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:55.056279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1336/2688] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:58.549387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:58.549930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:58.549930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:58.571931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] MFA Rules not processed for user `e2f423922c784da2be2d16c736a1ec16`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:17:58.573172 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:58.581505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1353/2689] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:17:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:17:59.174754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:59.176253 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:59.181794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:17:59.253688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730ef03-0616-4607-80c0-d0266d0b347e service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:59.265219 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:59.266102 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:59.266597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:59.267021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:59.269267 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:59.282114 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:59.283669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:59.296083 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1337/2690] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:17:59.858860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:17:59.869927 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb37fb2-0ada-4606-a58b-632f452988a6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:17:59.874148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:17:59.874914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:17:59.875237 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:17:59.875570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:17:59.877511 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:17:59.887535 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:17:59.889152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee18956-6dfb-4872-b2bb-9632bc04cb90 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:17:59.897605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1354/2691] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:17:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:00.794654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:00.797101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:00.813211 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6bbe6-2318-41b2-a38f-877d8f67aca5 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:00.818701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:00.819580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:00.819861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:00.820134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] PATH_INFO: `/v3/users/86049905c7214a49bdb4b4a4451e5809` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:00.821445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86049905c7214a49bdb4b4a4451e5809)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:00.823777 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:00.882130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86049905c7214a49bdb4b4a4451e5809'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:00.891185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86049905c7214a49bdb4b4a4451e5809'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:00.891985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86049905c7214a49bdb4b4a4451e5809'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:00.905985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86049905c7214a49bdb4b4a4451e5809'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:00.906722 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86049905c7214a49bdb4b4a4451e5809'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:00.922125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1338/2692] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:00 2022] DELETE /identity/v3/users/86049905c7214a49bdb4b4a4451e5809 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:00.926761 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:00.927282 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:00.927402 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:00.947291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:00.948605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:01.010671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1355/2693] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:01.020816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:01.022635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:01.086286 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3e19-f825-44cb-b8e0-b803c27907f6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:01.089891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:01.090602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:01.090918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:01.091191 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:01.100699 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:01.102147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:01.113365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1339/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:18:01.416606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:01.419295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:01.438046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609159-f4cb-4fbb-a8e6-796249578089 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:01.444522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:01.446314 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:01.446779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:01.447217 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] PATH_INFO: `/v3/projects/a86579199e244f59b6d8c86c4531d132` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:01.449123 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86579199e244f59b6d8c86c4531d132)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:01.451751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:01.466234 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:01.517052 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86579199e244f59b6d8c86c4531d132', 'request_id': 'req-ff2195d4-09ff-4c77-871f-a0b990fce0b3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.529102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86579199e244f59b6d8c86c4531d132', 'request_id': 'req-ff2195d4-09ff-4c77-871f-a0b990fce0b3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.538293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86579199e244f59b6d8c86c4531d132', 'request_id': 'req-ff2195d4-09ff-4c77-871f-a0b990fce0b3'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.554623 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] The token cache is being invalidate because project a86579199e244f59b6d8c86c4531d132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:01.555230 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.556064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.558591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1356/2695] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:01 2022] DELETE /identity/v3/projects/a86579199e244f59b6d8c86c4531d132 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:01.563033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:01.574771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965e60-d700-4f89-b894-73be3ff7c135 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:01.580095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:01.651612 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:01.652572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:01.652851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:01.653189 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] PATH_INFO: `/v3/users/73572f18ac7644b1ad0b169d795edca3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:01.654462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=73572f18ac7644b1ad0b169d795edca3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:01.656471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:01.714485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73572f18ac7644b1ad0b169d795edca3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.726244 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73572f18ac7644b1ad0b169d795edca3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.726979 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73572f18ac7644b1ad0b169d795edca3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.740606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:01.743033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:01.763359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73572f18ac7644b1ad0b169d795edca3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.764327 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73572f18ac7644b1ad0b169d795edca3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:01.780604 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1340/2696] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:01 2022] DELETE /identity/v3/users/73572f18ac7644b1ad0b169d795edca3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:01.843750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2195d4-09ff-4c77-871f-a0b990fce0b3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:01.912360 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:01.913075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:01.913419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:01.913752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:01.915676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:01.929073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:01.930736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd26aa-b0f1-4ef8-a8ae-44c456a0d9c2 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:01.942563 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1357/2697] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:18:02.481850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:02.483436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:02.551075 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06557-73e7-46fc-af6e-b7905678d1ec admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:02.555599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:02.556469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.556749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.557026 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] PATH_INFO: `/v3/projects/c9b9dede3a164565990728da672e1c6d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.558193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b9dede3a164565990728da672e1c6d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:02.560201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:02.593863 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:02.611654 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b9dede3a164565990728da672e1c6d', 'request_id': 'req-c7a12d5d-1209-4940-b7a3-7f30ab920a50'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:02.620416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b9dede3a164565990728da672e1c6d', 'request_id': 'req-c7a12d5d-1209-4940-b7a3-7f30ab920a50'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:02.626848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b9dede3a164565990728da672e1c6d', 'request_id': 'req-c7a12d5d-1209-4940-b7a3-7f30ab920a50'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:02.637733 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] The token cache is being invalidate because project c9b9dede3a164565990728da672e1c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:02.638022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:02.638732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7a12d5d-1209-4940-b7a3-7f30ab920a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:02.640407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1341/2698] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:02 2022] DELETE /identity/v3/projects/c9b9dede3a164565990728da672e1c6d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:02.645418 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.645708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.645984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.664003 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:02.665221 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:02.684024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:02.728586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1358/2699] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:02.736588 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.736881 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.737159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.757069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:02.758279 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:02.766763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1342/2700] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:02.771171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:02.781009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85da86c-1860-4ebc-b310-ad0e118c8859 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:02.785726 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:02.786417 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.786698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.786972 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.787729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:18:02.788042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:02.789634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:02.801368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1359/2701] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:18:02.805368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:02.814931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ea7af-5770-4acf-a94c-b281ec8fcf55 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:02.819737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:02.820397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.820677 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.820944 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.821637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:02.823611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:02.859415 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1343/2702] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:02 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:02.860644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:02.872256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844a3d5-72cd-4349-bdef-51d3570c632a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:02.930112 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:02.930878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.931104 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.931379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.932073 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:02.933995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:02.958604 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1360/2703] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:02 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:02.960120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:02.971242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca114a5-cd09-40df-918a-42c3523c6553 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:02.975717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:02.976367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:02.976645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:02.976914 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:02.977637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:02.979317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:02.989798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1344/2704] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:02.993806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.004029 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b110e-91e1-4071-80b9-3e065c450bb5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.008950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.009609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.009914 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.010186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.010945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.012748 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17209b7c-1cdc-4612-a76d-97b0b9d573c2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.023218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1361/2705] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:03.027366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.038966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d9af1-a108-4395-9edb-e0f313e1d376 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.044750 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.045632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.045910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.046183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] PATH_INFO: `/v3/projects/92c1416e71a5453790fe1517f94b1098/users/daae0127ca7b4ec78e28d5fe746e53cd/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.062185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c1416e71a5453790fe1517f94b1098, user_id=daae0127ca7b4ec78e28d5fe746e53cd, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.064876 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.082533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1345/2706] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:03 2022] PUT /identity/v3/projects/92c1416e71a5453790fe1517f94b1098/users/daae0127ca7b4ec78e28d5fe746e53cd/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:03.093027 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.093316 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.093581 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.120870 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] MFA Rules not processed for user `daae0127ca7b4ec78e28d5fe746e53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:03.122245 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:03.197176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1362/2707] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:03.201949 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.265629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59936061-b422-462f-bb51-b5681437b771 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.270673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.271342 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.271622 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.271894 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.272580 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.274522 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.303512 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1346/2708] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:03 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:03.307935 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.319592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb778f3-5899-4bbc-a6e8-c414015f93d1 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.332805 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:03.379577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.380249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.380534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.380812 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.381515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.383506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.407356 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1363/2709] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:03 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:03.411435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.421965 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dc7487-6350-4c78-80b2-17ad4b5d8fe4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.426737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.427386 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.427665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.427936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.428657 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.430259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.440927 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1347/2710] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:03.445048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.455189 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d888ad-ca3c-4ed3-a32b-8abe05345d12 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.460492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.461379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.461689 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.461963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] PATH_INFO: `/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f/users/4f93ab30d1ee4d1bb16a11c8b9ece777/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.478524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400e16d43d2341d6998dc8d51ee4bf9f, user_id=4f93ab30d1ee4d1bb16a11c8b9ece777, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.481311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.496696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1364/2711] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:03 2022] PUT /identity/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f/users/4f93ab30d1ee4d1bb16a11c8b9ece777/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:03.500977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.511299 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4c688-f1a9-4fb6-818b-d3b876882033 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.538664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:03.575393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.576124 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.576406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.576675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.577429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.579091 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518c9c7-4fda-47b8-9231-86ea11924c27 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.589515 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1348/2712] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:03.594004 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.605588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2c0c7-2fa1-41e5-ae33-4c6d07804a10 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.611638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.612524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.612804 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.613074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] PATH_INFO: `/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f/users/4f93ab30d1ee4d1bb16a11c8b9ece777/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.615435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=400e16d43d2341d6998dc8d51ee4bf9f, user_id=4f93ab30d1ee4d1bb16a11c8b9ece777, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.618080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.629886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1365/2713] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:03 2022] PUT /identity/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f/users/4f93ab30d1ee4d1bb16a11c8b9ece777/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:03.641207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.641494 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.641766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.670239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] MFA Rules not processed for user `4f93ab30d1ee4d1bb16a11c8b9ece777`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:03.671485 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:03.749159 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1349/2714] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:03 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:03.760829 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:03.762842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:03.824721 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f008aac-9d79-4c60-8867-7b0cb6994f80 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:03.829845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:03.830516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:03.830793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:03.831061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:03.840960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:03.842584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed728775-8506-480f-8385-4cf136cd1dd5 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:03.852308 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1366/2715] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:03 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:04.278602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:04.343382 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692786c-5d7f-4742-a934-01a8c80d0562 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:04.348734 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:04.349607 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:04.349886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:04.350155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] PATH_INFO: `/v3/users/daae0127ca7b4ec78e28d5fe746e53cd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:04.351418 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daae0127ca7b4ec78e28d5fe746e53cd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:04.353341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:04.408263 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daae0127ca7b4ec78e28d5fe746e53cd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:04.410765 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:04.417830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daae0127ca7b4ec78e28d5fe746e53cd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:04.418581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daae0127ca7b4ec78e28d5fe746e53cd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:04.431728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daae0127ca7b4ec78e28d5fe746e53cd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:04.432354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daae0127ca7b4ec78e28d5fe746e53cd'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:04.444995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1350/2716] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:04 2022] DELETE /identity/v3/users/daae0127ca7b4ec78e28d5fe746e53cd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:04.449624 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:04.450095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:04.450223 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:04.468993 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:04.470582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:04.536247 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1367/2717] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:04.546198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:04.547960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:04.608324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60791-abaa-480d-aa7b-48b6bf39efe5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:04.612468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:04.613116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:04.613395 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:04.613640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:04.625145 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:04.626675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:04.637430 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1351/2718] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:05.052420 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:05.053878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:05.066934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece44cae-f68e-46d9-9b98-b28a15db1b6f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:05.076142 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:05.077528 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.077919 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.078266 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] PATH_INFO: `/v3/projects/92c1416e71a5453790fe1517f94b1098` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.079652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c1416e71a5453790fe1517f94b1098)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:05.081807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:05.092995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:05.144394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c1416e71a5453790fe1517f94b1098', 'request_id': 'req-e018950e-3034-4765-8122-34f328283a09'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.154034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c1416e71a5453790fe1517f94b1098', 'request_id': 'req-e018950e-3034-4765-8122-34f328283a09'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.161079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c1416e71a5453790fe1517f94b1098', 'request_id': 'req-e018950e-3034-4765-8122-34f328283a09'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.173488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] The token cache is being invalidate because project 92c1416e71a5453790fe1517f94b1098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:05.174275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.175515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.182850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1368/2719] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:05 2022] DELETE /identity/v3/projects/92c1416e71a5453790fe1517f94b1098 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:05.183317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:05.193154 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f18a2-15c5-44d4-b0c7-f42f404a1327 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:05.259968 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:05.260868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.261147 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.261416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] PATH_INFO: `/v3/users/4f93ab30d1ee4d1bb16a11c8b9ece777` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.267563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f93ab30d1ee4d1bb16a11c8b9ece777)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:05.271946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:05.333823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f93ab30d1ee4d1bb16a11c8b9ece777'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.351469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f93ab30d1ee4d1bb16a11c8b9ece777'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.352252 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f93ab30d1ee4d1bb16a11c8b9ece777'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.367931 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f93ab30d1ee4d1bb16a11c8b9ece777'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.368739 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-232fb20b-2942-430f-b323-e67646c8c8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f93ab30d1ee4d1bb16a11c8b9ece777'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.369984 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:05.383827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1352/2720] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:05 2022] DELETE /identity/v3/users/4f93ab30d1ee4d1bb16a11c8b9ece777 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:18:05.664512 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:05.666181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:05.682371 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018950e-3034-4765-8122-34f328283a09 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:05.744013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:05.744887 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.745165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.745434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] PATH_INFO: `/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.746696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=400e16d43d2341d6998dc8d51ee4bf9f)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:05.748988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:05.800444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '400e16d43d2341d6998dc8d51ee4bf9f', 'request_id': 'req-03e44358-8627-44df-b898-d2bed4402dc8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.809091 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '400e16d43d2341d6998dc8d51ee4bf9f', 'request_id': 'req-03e44358-8627-44df-b898-d2bed4402dc8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.816207 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '400e16d43d2341d6998dc8d51ee4bf9f', 'request_id': 'req-03e44358-8627-44df-b898-d2bed4402dc8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.828079 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] The token cache is being invalidate because project 400e16d43d2341d6998dc8d51ee4bf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:05.828364 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.829086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-03e44358-8627-44df-b898-d2bed4402dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:05.830780 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1369/2721] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:05 2022] DELETE /identity/v3/projects/400e16d43d2341d6998dc8d51ee4bf9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:05.835890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.836420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.836420 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.856590 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:05.857943 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:05.920166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1353/2722] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:05.928296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.928737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.929017 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.949878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:05.951197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:05.960265 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1370/2723] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:05.964764 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:05.975231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4334676-8192-4a78-96d7-88b7dc0e9b8c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:05.980775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:05.981469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:05.981746 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:05.982015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:05.982848 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:18:05.983148 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:05.984867 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:05.996976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1354/2724] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:18:06.001445 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.011000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d373bc-7215-4722-ab17-cc5bd5c63b9d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.015936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.016607 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.016868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.017139 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.017827 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.019791 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.047466 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:06.056120 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1371/2725] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:06.060665 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.071408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abb547-c629-4412-8157-7d6540265f2d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.133188 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.133935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.134266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.134574 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.135324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.137481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.162549 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1355/2726] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:06.166936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.177730 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da462f-bae7-42f8-92b6-2413cf851263 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.182232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.182922 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.183201 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.183475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.184204 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.185846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.196450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1372/2727] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:06.200501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.210553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd9bb6e-c678-4c07-83dc-8095698c34c4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.215477 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.216128 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.216406 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.216685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.217525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.219533 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b64728-08ca-474f-a0ec-6e3bceb6253f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.229488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1356/2728] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:06.234335 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.249482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06725a00-6aad-496c-a51c-e51dbf4facd2 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.256203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.257075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.257347 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.257611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] PATH_INFO: `/v3/projects/296e9b2e41a9453cb7039f1f1811948b/users/154fec5f52ce4771add2e17014d97578/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.272984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=296e9b2e41a9453cb7039f1f1811948b, user_id=154fec5f52ce4771add2e17014d97578, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.275665 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.290496 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1373/2729] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:06 2022] PUT /identity/v3/projects/296e9b2e41a9453cb7039f1f1811948b/users/154fec5f52ce4771add2e17014d97578/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:06.302405 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.302700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.302982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.331143 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] MFA Rules not processed for user `154fec5f52ce4771add2e17014d97578`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:06.332366 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:06.379891 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:06.401698 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1357/2730] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:06.406295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.469415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09517ad-ca3a-4c73-bd40-302648d8e98a admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.475119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.475869 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.476718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.477061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.477840 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.479863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.510781 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1374/2731] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:06.514864 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.525701 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9bab5-bb41-4ae6-9da1-2e373f966bb6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.586926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.587644 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.588049 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.588313 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.589144 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.590955 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.612237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1358/2732] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:06.616315 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.627527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dd237-a586-4860-9816-102a1cfec0da admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.633289 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.633941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.634218 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.634516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.635259 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.636860 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.647675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1375/2733] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:06.651850 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.662432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de00faa-8c89-41a9-a510-90b841d66b94 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.668707 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d849-3d50-4948-b736-59336660062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.669620 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d849-3d50-4948-b736-59336660062f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.669924 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d849-3d50-4948-b736-59336660062f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.670152 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819d849-3d50-4948-b736-59336660062f admin admin] PATH_INFO: `/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3/users/4bf5eae65d6e41f1a4aa5fb314a04761/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.687234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d849-3d50-4948-b736-59336660062f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7593d21bf3e4dbbb4d2b2a0fff100e3, user_id=4bf5eae65d6e41f1a4aa5fb314a04761, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.689945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819d849-3d50-4948-b736-59336660062f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.708069 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1359/2734] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:06 2022] PUT /identity/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3/users/4bf5eae65d6e41f1a4aa5fb314a04761/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:06.712801 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.724971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc922bd-f08c-4789-8d49-9fcb7883680c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.786181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.786881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.787164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.787419 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.788162 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.789737 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0046d-f813-4384-b4f9-8014b5a72669 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.800250 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1376/2735] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:06.804619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d849-3d50-4948-b736-59336660062f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.815197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819d849-3d50-4948-b736-59336660062f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:06.819978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:06.820855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.821134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.821404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] PATH_INFO: `/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3/users/4bf5eae65d6e41f1a4aa5fb314a04761/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.823397 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7593d21bf3e4dbbb4d2b2a0fff100e3, user_id=4bf5eae65d6e41f1a4aa5fb314a04761, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:06.826029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:06.836354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1360/2736] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:06 2022] PUT /identity/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3/users/4bf5eae65d6e41f1a4aa5fb314a04761/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:06.846894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:06.847152 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:06.847464 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:06.874546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] MFA Rules not processed for user `4bf5eae65d6e41f1a4aa5fb314a04761`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:06.875745 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:06.901990 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:06.955611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1377/2737] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:06 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:06.967251 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:06.969774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:07.032610 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e359ca-baa7-4542-9f1a-f1ee45460288 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:07.037645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:07.038297 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:07.038614 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:07.038887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:07.048704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:07.050153 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:07.059686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1361/2738] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:06 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:07.143043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:07.153619 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21765b92-a829-4ec7-a272-dfd03ff9f621 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:07.157856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:07.158584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:07.158859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:07.159121 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:07.221252 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:07.222651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:07.231941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1378/2739] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:07.271732 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:07.273256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:07.328196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:07.375986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fbe17-8a31-4d24-8518-80a9e1a4d0b9 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:07.385810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:07.386511 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:07.386793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:07.387068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:07.388859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:07.401506 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:07.403156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38160e4-9ca5-4f4f-b7f4-419e01d93809 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:07.414248 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1362/2740] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:09.996106 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:10.059075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630555c-2529-48f8-bbce-9a7ebd886faa service cinder] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:10.063811 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:10.064751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:10.065094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:10.065461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] PATH_INFO: `/v3/users/154fec5f52ce4771add2e17014d97578` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:10.066778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=154fec5f52ce4771add2e17014d97578)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:10.068758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:10.119170 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '154fec5f52ce4771add2e17014d97578'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.128178 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '154fec5f52ce4771add2e17014d97578'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.128871 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '154fec5f52ce4771add2e17014d97578'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.133522 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:10.141865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '154fec5f52ce4771add2e17014d97578'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.142631 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '154fec5f52ce4771add2e17014d97578'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.155443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1379/2741] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:09 2022] DELETE /identity/v3/users/154fec5f52ce4771add2e17014d97578 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:10.160270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:10.160781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:10.160781 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:10.179040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:10.180229 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:10.243660 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1363/2742] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:10.254006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:10.255524 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:10.321309 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53399ea-0e3e-42d9-8286-ceb61a64fc7b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:10.324877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:10.325529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:10.325809 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:10.326082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:10.336306 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:10.337750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:10.348023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1380/2743] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:10.621754 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:10.623304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:10.635859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e17e5-9293-42f2-9aa1-f2f727b3287f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:10.641424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:10.642226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:10.642556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:10.642830 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] PATH_INFO: `/v3/projects/296e9b2e41a9453cb7039f1f1811948b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:10.644040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=296e9b2e41a9453cb7039f1f1811948b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:10.645957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:10.704051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '296e9b2e41a9453cb7039f1f1811948b', 'request_id': 'req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.714134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '296e9b2e41a9453cb7039f1f1811948b', 'request_id': 'req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.721860 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '296e9b2e41a9453cb7039f1f1811948b', 'request_id': 'req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.732916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] The token cache is being invalidate because project 296e9b2e41a9453cb7039f1f1811948b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:10.733203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.734064 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.735995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1364/2744] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:10 2022] DELETE /identity/v3/projects/296e9b2e41a9453cb7039f1f1811948b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:10.739863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:10.750224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490ee97-0176-41b1-9513-1094304363e3 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:10.808392 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:10.809270 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:10.809548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:10.809818 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] PATH_INFO: `/v3/users/4bf5eae65d6e41f1a4aa5fb314a04761` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:10.811041 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf5eae65d6e41f1a4aa5fb314a04761)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:10.813009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:10.866960 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf5eae65d6e41f1a4aa5fb314a04761'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.875863 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf5eae65d6e41f1a4aa5fb314a04761'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.876523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf5eae65d6e41f1a4aa5fb314a04761'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.888457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf5eae65d6e41f1a4aa5fb314a04761'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.889086 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aba727f5-8b51-4743-a56c-cb11645a33d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf5eae65d6e41f1a4aa5fb314a04761'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:10.897154 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:10.903183 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1381/2745] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:10 2022] DELETE /identity/v3/users/4bf5eae65d6e41f1a4aa5fb314a04761 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:11.182249 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.183843 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:11.197807 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbd6b2-cb92-48d6-a8d1-2daa4b93b2a3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.244302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:11.257681 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.258576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.258853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.259121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] PATH_INFO: `/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.260318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7593d21bf3e4dbbb4d2b2a0fff100e3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.262230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.313832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7593d21bf3e4dbbb4d2b2a0fff100e3', 'request_id': 'req-c360479a-821e-4d0d-aebb-07e60f5d1721'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:11.323407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7593d21bf3e4dbbb4d2b2a0fff100e3', 'request_id': 'req-c360479a-821e-4d0d-aebb-07e60f5d1721'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:11.329784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7593d21bf3e4dbbb4d2b2a0fff100e3', 'request_id': 'req-c360479a-821e-4d0d-aebb-07e60f5d1721'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:11.340100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] The token cache is being invalidate because project e7593d21bf3e4dbbb4d2b2a0fff100e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:11.340381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:11.341088 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c360479a-821e-4d0d-aebb-07e60f5d1721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:11.342666 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1365/2746] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:11 2022] DELETE /identity/v3/projects/e7593d21bf3e4dbbb4d2b2a0fff100e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:11.348469 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.348998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.348998 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.366881 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:11.368077 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:11.427239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1382/2747] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:11.436207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.436599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.437302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.456981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:11.458204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:11.467619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1366/2748] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:11.472877 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.486557 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d49c72-be15-46a0-bab0-3949915706f7 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.491373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.492028 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.492297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.492567 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.493285 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:18:11.493589 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.495210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.506796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1383/2749] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:18:11.511240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.521426 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d20b4f-6aa1-4b23-889a-b18310dc7a6b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.527098 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.527753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.528031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.528321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.529005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.531002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.559413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1367/2750] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:11.564359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.578825 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced43544-5015-4ca6-8233-cd6af57096a8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.636711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.637378 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.637655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.637926 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.638662 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.640584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.663975 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1384/2751] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:11.668300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.679318 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f135f3-0718-45d5-adbe-2d36061235ab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.684187 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.684849 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.685132 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.685407 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.686142 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.687916 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.698287 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1368/2752] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:11.702228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.712322 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10b068-b15c-4fa6-a79f-37202a21846c admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.717646 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.718296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.718606 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.718878 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.719942 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.721525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623f6a5a-9736-4f4d-a2fb-0a046c85ca7e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.731729 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1385/2753] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:11.735996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.746050 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cb623-8ba7-43e3-b90a-623c4fd7b762 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.751109 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.751991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.752271 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.752540 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] PATH_INFO: `/v3/projects/8155b4f7662446cb94a24ec36161d65d/users/46f7dd3370404c6895c1d5d6d78399ac/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.767436 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8155b4f7662446cb94a24ec36161d65d, user_id=46f7dd3370404c6895c1d5d6d78399ac, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.770094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:11.788743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1369/2754] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:11 2022] PUT /identity/v3/projects/8155b4f7662446cb94a24ec36161d65d/users/46f7dd3370404c6895c1d5d6d78399ac/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:11.802874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.803164 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.803441 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.832594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] MFA Rules not processed for user `46f7dd3370404c6895c1d5d6d78399ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:11.833823 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:11.893191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:11.904784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1386/2755] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:11.909170 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:11.975940 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b1e79d-9769-4d28-82f4-6397afef5ab1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:11.981175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:11.981858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:11.982116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:11.982434 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:11.983237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:11.985493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.009728 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:12.018213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1370/2756] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:11 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:12.022493 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.033889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b37030-e5b5-4dec-a149-bcc40890845c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.102222 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.102934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.103197 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.103468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.104157 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.106056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.131093 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1387/2757] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:12 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:12.135766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.147971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89cf72-ba93-4d84-b380-ceb0f4340276 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.153272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.153935 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.154239 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.154553 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.155276 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.156981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.167841 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1371/2758] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:12.172054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.185331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a44b-5598-4613-88b4-8cc400d53d6b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.193146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.194055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.194383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.194719 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] PATH_INFO: `/v3/projects/8c9c306e7bc94985965bd213273377bb/users/2b4174e9df5f40a6a70b3683fb5b8f3f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.211244 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9c306e7bc94985965bd213273377bb, user_id=2b4174e9df5f40a6a70b3683fb5b8f3f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.214080 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.229832 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1388/2759] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:12 2022] PUT /identity/v3/projects/8c9c306e7bc94985965bd213273377bb/users/2b4174e9df5f40a6a70b3683fb5b8f3f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:18:12.233898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.244519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b9052e-b956-4dd2-8bfa-f6d5ffc39db8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.302579 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.303548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.303829 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.304100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.304821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.306454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2ea6a-34ce-4ada-b443-498b645185ed admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.319910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1372/2760] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:18:12.321683 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.333009 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e42bbf-0df5-4efd-867d-792e12fd3709 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.338488 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.339384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.339672 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.339945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] PATH_INFO: `/v3/projects/8c9c306e7bc94985965bd213273377bb/users/2b4174e9df5f40a6a70b3683fb5b8f3f/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.341956 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9c306e7bc94985965bd213273377bb, user_id=2b4174e9df5f40a6a70b3683fb5b8f3f, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.344995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.356269 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1389/2761] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:12 2022] PUT /identity/v3/projects/8c9c306e7bc94985965bd213273377bb/users/2b4174e9df5f40a6a70b3683fb5b8f3f/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:18:12.366655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.366991 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.367266 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.394592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] MFA Rules not processed for user `2b4174e9df5f40a6a70b3683fb5b8f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:12.395819 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:12.467150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1373/2762] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:12.478404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:12.480735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:12.538290 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427b13a-7fd6-4f1b-8223-db9521ecbe05 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:12.542075 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:12.542751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:12.543054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:12.543324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:12.552858 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:12.554337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b94ae-71ff-40ea-81eb-2ce42667847d service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:12.563999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1390/2763] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:13.983282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:14.048958 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41043cc-2708-4254-bf85-9a675868f142 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:14.054801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:14.055690 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.055969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.056241 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] PATH_INFO: `/v3/users/46f7dd3370404c6895c1d5d6d78399ac` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.057593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f7dd3370404c6895c1d5d6d78399ac)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:14.059591 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:14.073846 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:14.115509 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f7dd3370404c6895c1d5d6d78399ac'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.124156 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f7dd3370404c6895c1d5d6d78399ac'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.124852 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f7dd3370404c6895c1d5d6d78399ac'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.138242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f7dd3370404c6895c1d5d6d78399ac'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.138926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ee4130f8-dba8-4941-ac72-5b1419986f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f7dd3370404c6895c1d5d6d78399ac'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.150816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1374/2764] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:13 2022] DELETE /identity/v3/users/46f7dd3370404c6895c1d5d6d78399ac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:14.155462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.155962 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.156064 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.176045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:14.177185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.206421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.206705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.206978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.224890 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] MFA Rules not processed for user `e2f423922c784da2be2d16c736a1ec16`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:14.226120 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.229668 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:14.245751 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1391/2765] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:14.255655 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:14.257251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.285900 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1375/2766] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:14.320406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b536b9-548e-4e03-8dfb-3a8f56dc674c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:14.323992 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:14.324635 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.324911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.325181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.335228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:14.336736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:14.355514 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1392/2767] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:18:14.537369 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:14.538851 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.552096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d339c5-6164-4aaf-acaf-9e3581267c27 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:14.556387 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:14.557034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.557314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.557618 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.559645 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.569986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:14.571471 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:14.579920 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1376/2768] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:14.716409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:14.717936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:14.731920 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d675d85-ac82-4c49-bba3-2ae0720fccf4 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:14.737225 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:14.738094 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.738415 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.738688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] PATH_INFO: `/v3/projects/8155b4f7662446cb94a24ec36161d65d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.739904 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8155b4f7662446cb94a24ec36161d65d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:14.741844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:14.795647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8155b4f7662446cb94a24ec36161d65d', 'request_id': 'req-ec32ebf4-f2b1-4173-9493-179614d286e6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.805735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8155b4f7662446cb94a24ec36161d65d', 'request_id': 'req-ec32ebf4-f2b1-4173-9493-179614d286e6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.812240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8155b4f7662446cb94a24ec36161d65d', 'request_id': 'req-ec32ebf4-f2b1-4173-9493-179614d286e6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.823611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] The token cache is being invalidate because project 8155b4f7662446cb94a24ec36161d65d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:14.823901 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.824605 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.826298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1393/2769] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:14 2022] DELETE /identity/v3/projects/8155b4f7662446cb94a24ec36161d65d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:14.831121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:14.842422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673523b6-9050-4cfc-808e-4fe7523d8443 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:14.902989 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:14.903886 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:14.904166 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:14.904440 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] PATH_INFO: `/v3/users/2b4174e9df5f40a6a70b3683fb5b8f3f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:14.905649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4174e9df5f40a6a70b3683fb5b8f3f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:14.908055 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:14.935380 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:14.966105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4174e9df5f40a6a70b3683fb5b8f3f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.975753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4174e9df5f40a6a70b3683fb5b8f3f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:14.978262 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4174e9df5f40a6a70b3683fb5b8f3f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.005292 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4174e9df5f40a6a70b3683fb5b8f3f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.006146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5563b9c3-d161-4a99-a987-16f8916678e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4174e9df5f40a6a70b3683fb5b8f3f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.021570 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1377/2770] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:14 2022] DELETE /identity/v3/users/2b4174e9df5f40a6a70b3683fb5b8f3f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:18:15.314074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.315645 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:15.329492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32ebf4-f2b1-4173-9493-179614d286e6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.397305 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.398210 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.399566 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.399909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] PATH_INFO: `/v3/projects/8c9c306e7bc94985965bd213273377bb` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.401156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9c306e7bc94985965bd213273377bb)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.403174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.420609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:15.452650 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9c306e7bc94985965bd213273377bb', 'request_id': 'req-dc79c73d-219a-4f09-91e0-896f0e761f41'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.461523 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9c306e7bc94985965bd213273377bb', 'request_id': 'req-dc79c73d-219a-4f09-91e0-896f0e761f41'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.467974 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9c306e7bc94985965bd213273377bb', 'request_id': 'req-dc79c73d-219a-4f09-91e0-896f0e761f41'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.479868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] The token cache is being invalidate because project 8c9c306e7bc94985965bd213273377bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:15.480416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.481260 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc79c73d-219a-4f09-91e0-896f0e761f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:15.483066 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1394/2771] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:15 2022] DELETE /identity/v3/projects/8c9c306e7bc94985965bd213273377bb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:15.488839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.489133 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.489411 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.508978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:15.510208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:15.577568 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1378/2772] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:15.585799 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.586084 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.586391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.606006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:15.607263 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:15.616058 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1395/2773] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:15.620504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.630674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72bc448-38da-4e4a-9cc1-cd05ca3217e9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.635741 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.636445 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.636724 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.637002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.637726 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:18:15.638033 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.639683 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.651519 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1379/2774] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:18:15.658038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.667443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ff2d3-c986-4c6a-ab3a-381684aa7a40 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.673433 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.674055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.674363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.674664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.675352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.677241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.705532 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1396/2775] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:15 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:15.710493 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.724201 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67558a-62c9-41aa-be76-7cb2ea978a82 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.785174 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.785874 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.786146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.786448 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.787136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.789059 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.814755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1380/2776] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:15 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:15.816108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.827074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df001477-f363-415f-a13b-c8924a9798f5 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.832959 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.833560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.833898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.834249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.835038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.836679 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.848330 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1397/2777] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:15.852624 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.863014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d3d13-f36e-4e24-8601-5492cf9c15d8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.867635 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.868350 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.868633 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.868903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.870031 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.871410 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7a6bc-1b4a-4a5f-ad7a-4444f9a0b6eb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.882836 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1381/2778] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:15.887411 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:15.898101 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964f8f2-8c83-4111-a6c3-756fb7fcd6ff admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:15.903506 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:15.904428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:15.904710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:15.904984 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] PATH_INFO: `/v3/projects/56f225808d1b4dc88ace2d074ee80afb/users/6a893a412a8b4e1ebedb158e54750ef6/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:15.921950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f225808d1b4dc88ace2d074ee80afb, user_id=6a893a412a8b4e1ebedb158e54750ef6, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:15.924944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:15.941010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1398/2779] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:15 2022] PUT /identity/v3/projects/56f225808d1b4dc88ace2d074ee80afb/users/6a893a412a8b4e1ebedb158e54750ef6/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:16.009343 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.009957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.009957 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.024810 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:16.044993 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] MFA Rules not processed for user `6a893a412a8b4e1ebedb158e54750ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:16.047022 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:16.128444 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1382/2780] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:16.133800 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.199873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ef9cc-4153-44eb-af7f-82fc3d2f7d72 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.205249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.206095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.206475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.206779 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.207468 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.209424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.239136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1399/2781] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:16 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:18:16.243431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.254431 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0def4e2-f49c-4faf-8524-065d7448f665 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.317915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.318731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.319016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.319317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.320005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.321938 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.346217 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1383/2782] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:16 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:16.350341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.361454 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d464b34-a14b-4ffa-a2b5-6aefc4b4b365 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.366884 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.367582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.367861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.368146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.368875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.370553 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.382546 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1400/2783] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:16.386713 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.398099 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb9e28-cf2b-4b0c-9178-749b05bd9a51 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.403135 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.404176 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.404469 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.404721 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] PATH_INFO: `/v3/projects/bef8ae9469784fcea20553bc730e7577/users/53a9916a70a44b5cbe02bcdb8c51d8f7/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.422035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef8ae9469784fcea20553bc730e7577, user_id=53a9916a70a44b5cbe02bcdb8c51d8f7, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.425003 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.439651 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1384/2784] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:16 2022] PUT /identity/v3/projects/bef8ae9469784fcea20553bc730e7577/users/53a9916a70a44b5cbe02bcdb8c51d8f7/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:16.443609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.453509 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972a2cf-7ef1-4452-9208-0812bad2ab25 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.464735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:16.513450 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.514136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.514424 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.514700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.515421 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.517046 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d968463-8094-4f0f-a825-31b65aaf6722 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.527490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1401/2785] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:16.531601 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.542521 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478437c8-c3cd-4e8a-b876-88bb96f824f2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.547063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.547842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.548121 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.548565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] PATH_INFO: `/v3/projects/bef8ae9469784fcea20553bc730e7577/users/53a9916a70a44b5cbe02bcdb8c51d8f7/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.551816 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef8ae9469784fcea20553bc730e7577, user_id=53a9916a70a44b5cbe02bcdb8c51d8f7, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.553041 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.563939 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1385/2786] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:16 2022] PUT /identity/v3/projects/bef8ae9469784fcea20553bc730e7577/users/53a9916a70a44b5cbe02bcdb8c51d8f7/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:16.574067 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.574346 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.574670 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.605701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] MFA Rules not processed for user `53a9916a70a44b5cbe02bcdb8c51d8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:16.606964 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:16.677795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1402/2787] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:16.688947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:16.690896 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:16.745334 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:16.749321 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c24717-f56e-4316-b048-0e506b8edc24 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:16.755080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:16.755763 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:16.756173 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:16.756455 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:16.815870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:16.817442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c084da-953c-43a5-b132-c4bf2140ddae service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:16.826817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1386/2788] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:22.525664 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:22.527275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:22.584384 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0508644-e514-44b4-8634-206c9c86bcfe None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:22.588708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:22.589568 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:22.589856 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:22.590130 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:22.627712 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:22.629971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09e4bf-e983-4241-ad2b-def461c8f743 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:22.641224 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1403/2789] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:26.822404 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=26 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:32.635911 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=81 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:56.826312 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:56.826941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:56.826941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:56.839040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:56.839708 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:56.839814 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:56.846058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] MFA Rules not processed for user `1c4aa9e4d27c4c2ea4b49ab188f841da`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:56.847378 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:56.858095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:56.859303 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:56.904130 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1387/2790] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:56.928141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1404/2791] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:18:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:56.984048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:56.985554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:57.063854 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a32fd-7723-40e2-a120-28be379d00f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:57.068813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:57.069468 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:57.069751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:57.070025 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:57.079030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:57.080653 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62f75c-48fd-44a5-be1b-8c11117381e6 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:57.091966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1388/2792] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:18:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:18:57.111907 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:57.286175 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233950e-75e7-4148-a262-5c68876db0b5 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:57.294184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:57.296950 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:57.297555 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:57.298078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] PATH_INFO: `/v3/users/6a893a412a8b4e1ebedb158e54750ef6` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:57.300550 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a893a412a8b4e1ebedb158e54750ef6)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:57.304594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:57.381432 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a893a412a8b4e1ebedb158e54750ef6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:57.400693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a893a412a8b4e1ebedb158e54750ef6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:57.402146 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a893a412a8b4e1ebedb158e54750ef6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:57.423055 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a893a412a8b4e1ebedb158e54750ef6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:57.424585 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a893a412a8b4e1ebedb158e54750ef6'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:57.437932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:57.453291 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1405/2793] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:57 2022] DELETE /identity/v3/users/6a893a412a8b4e1ebedb158e54750ef6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:57.458774 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:57.459295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:57.459845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:57.510034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:57.512029 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:57.592282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1389/2794] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:57.611885 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:57.613494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:57.769108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260165e-8b22-42a2-87a4-bdc070bcb5c6 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:57.774838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:57.775837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:57.776345 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:57.776826 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:57.788987 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:57.794383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:57.812783 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1406/2795] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:18:58.250833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:58.252889 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:58.273596 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10a47b-da09-44cc-aab4-d728236b38ea None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:58.294139 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:58.295374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:58.299354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:58.300095 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] PATH_INFO: `/v3/projects/56f225808d1b4dc88ace2d074ee80afb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:58.302556 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f225808d1b4dc88ace2d074ee80afb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:58.307345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:58.315563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:58.379583 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f225808d1b4dc88ace2d074ee80afb', 'request_id': 'req-d53da468-99fd-457b-ba09-d151d03679a8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.392115 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f225808d1b4dc88ace2d074ee80afb', 'request_id': 'req-d53da468-99fd-457b-ba09-d151d03679a8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.415451 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f225808d1b4dc88ace2d074ee80afb', 'request_id': 'req-d53da468-99fd-457b-ba09-d151d03679a8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.429855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] The token cache is being invalidate because project 56f225808d1b4dc88ace2d074ee80afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:58.430298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.431080 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.433488 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1390/2796] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:58 2022] DELETE /identity/v3/projects/56f225808d1b4dc88ace2d074ee80afb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:58.438438 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:58.450641 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a2183-2d10-418b-b9fc-bea67de5ee96 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:58.545295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:58.546232 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:58.546573 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:58.546886 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] PATH_INFO: `/v3/users/53a9916a70a44b5cbe02bcdb8c51d8f7` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:58.548416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a9916a70a44b5cbe02bcdb8c51d8f7)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:58.550518 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:58.618467 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a9916a70a44b5cbe02bcdb8c51d8f7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.628690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a9916a70a44b5cbe02bcdb8c51d8f7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.629634 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a9916a70a44b5cbe02bcdb8c51d8f7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.646519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a9916a70a44b5cbe02bcdb8c51d8f7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.647854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-dc4a937e-8572-4ca3-a19c-d7d4d864d532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a9916a70a44b5cbe02bcdb8c51d8f7'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:58.662045 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1407/2797] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:18:58 2022] DELETE /identity/v3/users/53a9916a70a44b5cbe02bcdb8c51d8f7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:58.984576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:58.986039 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:59.002915 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53da468-99fd-457b-ba09-d151d03679a8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.106609 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.107500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.107784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.108063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] PATH_INFO: `/v3/projects/bef8ae9469784fcea20553bc730e7577` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.109278 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef8ae9469784fcea20553bc730e7577)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.111902 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.206432 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef8ae9469784fcea20553bc730e7577', 'request_id': 'req-cea5ea3e-858e-41c0-a6fd-56c0b784f332'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:59.217119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef8ae9469784fcea20553bc730e7577', 'request_id': 'req-cea5ea3e-858e-41c0-a6fd-56c0b784f332'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:59.225966 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef8ae9469784fcea20553bc730e7577', 'request_id': 'req-cea5ea3e-858e-41c0-a6fd-56c0b784f332'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:59.239157 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] The token cache is being invalidate because project bef8ae9469784fcea20553bc730e7577 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:18:59.239577 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:59.240197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cea5ea3e-858e-41c0-a6fd-56c0b784f332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:18:59.247273 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1391/2798] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:18:58 2022] DELETE /identity/v3/projects/bef8ae9469784fcea20553bc730e7577 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:59.247889 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.248191 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.248428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.252227 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:59.343239 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:59.344924 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:59.426023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1408/2799] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:18:59.435956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19753-2849-4909-9015-db940a36b652 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.436552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19753-2849-4909-9015-db940a36b652 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.436552 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19753-2849-4909-9015-db940a36b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.457676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-39d19753-2849-4909-9015-db940a36b652 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:59.458969 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-39d19753-2849-4909-9015-db940a36b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:59.472322 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1392/2800] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:18:59.473363 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.488675 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666634f-ad86-4a94-8d5b-70c40a8e30d9 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.501560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.502376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.502994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.502994 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] PATH_INFO: `/v3/domains` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.503750 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80101) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:18:59.504054 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.505588 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.533344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1409/2801] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:18:59.537525 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d19753-2849-4909-9015-db940a36b652 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.557032 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d19753-2849-4909-9015-db940a36b652 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.568474 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.569162 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.569442 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.569715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.570467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.572462 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.588464 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:18:59.616070 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1393/2802] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:18:59 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:59.617644 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.629525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd83b5-f8d8-4329-8db0-8628af062624 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.696970 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.697701 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.698190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.698929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.699586 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.701658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.732966 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1410/2803] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:18:59 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 13:18:59.734467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.746629 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42d872-ffc0-47c0-955b-1a2eb1f617de admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.751639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.752314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.752594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.752870 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.753611 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.755264 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.770424 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1394/2804] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:18:59.774248 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.786016 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90162a-4fcb-4a2b-a87a-a960f88532a3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.790777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.791444 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.791723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.792001 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.793090 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.794752 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61235561-90f4-4b01-86f3-b424ef308dfc admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.810579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1411/2805] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:18:59.814481 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:18:59.826090 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5773f4-9b15-44c7-8765-6044954aabbc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:18:59.831308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:18:59.832311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.833749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.834103 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] PATH_INFO: `/v3/projects/8140a7423cd043b0befb16e2d2f5891c/users/513969b9e6124343a8e368c6e3b9e6d7/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.851213 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8140a7423cd043b0befb16e2d2f5891c, user_id=513969b9e6124343a8e368c6e3b9e6d7, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:18:59.853946 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:18:59.882571 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1395/2806] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:18:59 2022] PUT /identity/v3/projects/8140a7423cd043b0befb16e2d2f5891c/users/513969b9e6124343a8e368c6e3b9e6d7/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:18:59.885151 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:18:59.885698 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:18:59.886427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:18:59.925074 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] MFA Rules not processed for user `513969b9e6124343a8e368c6e3b9e6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:18:59.927138 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:18:59.978430 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:00.024435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1412/2807] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:18:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:00.029454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.127905 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16feff-99d5-4b5c-912f-d62fcecec47c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.134240 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.135005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.135347 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.135634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.137231 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.139439 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.179000 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1396/2808] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:19:00 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:00.180300 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.197033 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c117ba-5f0b-4370-aa50-542ddec9646a None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.344351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.345352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.346640 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.347048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] PATH_INFO: `/v3/users` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.347905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.350861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.385341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1413/2809] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:19:00 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:00.389626 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.405413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84fedc-b8c6-4150-919d-adfaff6073d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.424584 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.425435 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.425787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.426127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.428641 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.430673 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.453974 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1397/2810] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:00.457909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.488516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed34459-99a7-42f3-993c-46524114cf77 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.512409 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.513476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.513875 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.514249 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] PATH_INFO: `/v3/projects/404ff99f0323416f9a5dd072ef55c0df/users/68c4d0f8af1d4e93be10d927ace9eb98/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.542803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=404ff99f0323416f9a5dd072ef55c0df, user_id=68c4d0f8af1d4e93be10d927ace9eb98, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.550868 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.584611 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1414/2811] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:00 2022] PUT /identity/v3/projects/404ff99f0323416f9a5dd072ef55c0df/users/68c4d0f8af1d4e93be10d927ace9eb98/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:00.588817 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.599972 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def645e0-079f-4785-ab9a-e903a8fac6bb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.671516 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.671813 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.672096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.672384 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.673419 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.674952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0dd593-4dd1-46f5-bb4b-197ec87d98c8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.686308 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1398/2812] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:00.691010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.705558 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2865b8-7b51-4700-ae72-a27aa9fd7c99 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:00.710824 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:00.711905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.712182 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.712476 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] PATH_INFO: `/v3/projects/404ff99f0323416f9a5dd072ef55c0df/users/68c4d0f8af1d4e93be10d927ace9eb98/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.715457 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=404ff99f0323416f9a5dd072ef55c0df, user_id=68c4d0f8af1d4e93be10d927ace9eb98, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:00.719156 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:00.746228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1415/2813] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:00 2022] PUT /identity/v3/projects/404ff99f0323416f9a5dd072ef55c0df/users/68c4d0f8af1d4e93be10d927ace9eb98/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:00.747416 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:00.747708 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:00.747985 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:00.783550 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] MFA Rules not processed for user `68c4d0f8af1d4e93be10d927ace9eb98`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:00.785281 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:00.815771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:00.904839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1399/2814] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:00.912955 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:00.917038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:01.006583 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f925f-97f5-4ae9-be54-9fb379cab8a3 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:01.011176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:01.012039 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:01.012326 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:01.012943 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:01.026148 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:01.028267 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f6854a-7592-4dcf-b828-2fc6a7edda30 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:01.055048 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1416/2815] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:19:01.637786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:01.737357 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f4ea9-834b-4274-a602-749b5b092be5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:01.742107 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:01.743453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:01.743453 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:01.743691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] PATH_INFO: `/v3/users/513969b9e6124343a8e368c6e3b9e6d7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:01.744925 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] RBAC: Authorizing `identity:delete_user(user_id=513969b9e6124343a8e368c6e3b9e6d7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:01.747021 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:01.834895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513969b9e6124343a8e368c6e3b9e6d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:01.845542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513969b9e6124343a8e368c6e3b9e6d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:01.854928 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513969b9e6124343a8e368c6e3b9e6d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:01.873127 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513969b9e6124343a8e368c6e3b9e6d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:01.873866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513969b9e6124343a8e368c6e3b9e6d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:01.889749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1400/2816] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:19:01 2022] DELETE /identity/v3/users/513969b9e6124343a8e368c6e3b9e6d7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:01.894777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:01.895327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:01.895436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:01.915767 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:01.916938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:01.972294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:02.068516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1417/2817] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:02.078879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:02.080371 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:02.140226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:02.169952 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ecca0-6163-4399-9ef3-630a2e0ac77c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:02.174562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:02.175365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:02.175745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:02.176129 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:02.188800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:02.190335 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:02.203100 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1401/2818] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:02.540278 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:02.542754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:02.595025 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e507e5-497f-4e84-9d48-b2ce585dabbe None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:02.601116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:02.602264 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:02.602652 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:02.602995 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] PATH_INFO: `/v3/projects/8140a7423cd043b0befb16e2d2f5891c` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:02.604288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8140a7423cd043b0befb16e2d2f5891c)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:02.606373 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:02.671784 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8140a7423cd043b0befb16e2d2f5891c', 'request_id': 'req-50edf261-c967-4925-bfac-b96fd2e3b144'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.682777 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8140a7423cd043b0befb16e2d2f5891c', 'request_id': 'req-50edf261-c967-4925-bfac-b96fd2e3b144'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.690482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8140a7423cd043b0befb16e2d2f5891c', 'request_id': 'req-50edf261-c967-4925-bfac-b96fd2e3b144'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.702405 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] The token cache is being invalidate because project 8140a7423cd043b0befb16e2d2f5891c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:19:02.702718 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.703576 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.705638 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1418/2819] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:19:02 2022] DELETE /identity/v3/projects/8140a7423cd043b0befb16e2d2f5891c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:19:02.709648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:02.728048 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a95588-512f-4c5c-bc81-91e61441f21a service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:02.796175 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:02.797237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:02.797587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:02.797930 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] PATH_INFO: `/v3/users/68c4d0f8af1d4e93be10d927ace9eb98` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:02.799225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c4d0f8af1d4e93be10d927ace9eb98)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:02.801593 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:02.867331 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c4d0f8af1d4e93be10d927ace9eb98'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.879352 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c4d0f8af1d4e93be10d927ace9eb98'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.880413 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c4d0f8af1d4e93be10d927ace9eb98'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.895501 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c4d0f8af1d4e93be10d927ace9eb98'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.896709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e67c543e-47b8-422b-89cf-f31926c728f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c4d0f8af1d4e93be10d927ace9eb98'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:02.919539 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1402/2820] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:19:02 2022] DELETE /identity/v3/users/68c4d0f8af1d4e93be10d927ace9eb98 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:03.486873 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:03.490078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:03.524697 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edf261-c967-4925-bfac-b96fd2e3b144 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:03.618510 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:03.619564 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:03.619850 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:03.620427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] PATH_INFO: `/v3/projects/404ff99f0323416f9a5dd072ef55c0df` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:03.621676 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] RBAC: Authorizing `identity:delete_project(project_id=404ff99f0323416f9a5dd072ef55c0df)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:03.623673 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:03.715773 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '404ff99f0323416f9a5dd072ef55c0df', 'request_id': 'req-90717007-5c5c-447a-a6ae-830a015b14be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:03.721185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:03.729772 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '404ff99f0323416f9a5dd072ef55c0df', 'request_id': 'req-90717007-5c5c-447a-a6ae-830a015b14be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:03.740095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '404ff99f0323416f9a5dd072ef55c0df', 'request_id': 'req-90717007-5c5c-447a-a6ae-830a015b14be'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:03.756861 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] The token cache is being invalidate because project 404ff99f0323416f9a5dd072ef55c0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:19:03.757136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:03.757813 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-90717007-5c5c-447a-a6ae-830a015b14be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:03.760383 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1419/2821] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:19:03 2022] DELETE /identity/v3/projects/404ff99f0323416f9a5dd072ef55c0df => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:03.766261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:03.766879 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:03.767024 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:03.786887 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:03.788179 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:03.855663 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:03.867743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1403/2822] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:19:03.875854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:03.876311 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:03.876692 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:03.897969 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:03.899240 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:03.907724 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1420/2823] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:19:03.911998 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:03.923559 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3934321-7b67-4a96-9899-710c151c50fb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:03.928827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:03.929513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:03.929814 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:03.930074 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:03.930898 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:19:03.931202 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:03.932961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:03.950305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1404/2824] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 13:19:03.955932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:03.966465 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763904e-c5ab-437e-be75-709ca74c454c None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:03.971986 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:03.972796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:03.973144 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:03.973486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:03.974284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:03.976436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.009707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1421/2825] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:19:03 2022] POST /identity/v3/projects => generated 359 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:04.014314 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.035630 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf800f-7a49-4912-81a5-6005890d497c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.106499 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.107208 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.107587 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.107909 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.109381 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.112060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.187742 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1405/2826] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:19:04 2022] POST /identity/v3/users => generated 280 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:04.191938 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.204682 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60e52b-35be-4b98-872a-72b403a5f846 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.210609 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.211329 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.211671 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.212007 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.212845 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.214905 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.227842 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1422/2827] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:04.232365 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.244832 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6fe7b-ab57-4c9b-b873-ab931f770875 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.253871 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.254735 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.255116 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.255454 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.256492 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.258284 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806c6f9-7ef2-48c1-b6cd-a3285ec7b85a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.281259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1406/2828] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:04.285262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.300548 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde560c-213d-41a8-92ac-32c9916886e0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.309352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.310365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.310715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.311062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] PATH_INFO: `/v3/projects/3c29a8669ef14913a64ddfdcc3fcdd09/users/38bf0a342fc9441a95bd7087ac7f64a7/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.343793 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c29a8669ef14913a64ddfdcc3fcdd09, user_id=38bf0a342fc9441a95bd7087ac7f64a7, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.348274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.375082 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1423/2829] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:04 2022] PUT /identity/v3/projects/3c29a8669ef14913a64ddfdcc3fcdd09/users/38bf0a342fc9441a95bd7087ac7f64a7/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:19:04.385204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.385498 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.386038 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.437359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] MFA Rules not processed for user `38bf0a342fc9441a95bd7087ac7f64a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:04.438801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:04.521967 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1407/2830] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:04.559536 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.562134 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:04.642341 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc9a6a-3ab5-473c-b132-89b72848b5f4 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.646475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.647165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.647504 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.647797 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.658212 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.659716 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82190714-d57f-4ef2-9a42-ccde46046bf2 service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.670070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1424/2831] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 13:19:04.689827 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:04.760646 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:04.779796 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37892ee-e38e-4495-8fef-33e1e5e0c086 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:04.784565 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:04.785458 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.785823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.786015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] PATH_INFO: `/v3/users/38bf0a342fc9441a95bd7087ac7f64a7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.787341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] RBAC: Authorizing `identity:delete_user(user_id=38bf0a342fc9441a95bd7087ac7f64a7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:04.789309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:04.846532 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38bf0a342fc9441a95bd7087ac7f64a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:04.860257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38bf0a342fc9441a95bd7087ac7f64a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:04.861227 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38bf0a342fc9441a95bd7087ac7f64a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:04.879204 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38bf0a342fc9441a95bd7087ac7f64a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:04.879937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38bf0a342fc9441a95bd7087ac7f64a7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:04.898597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1408/2832] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:19:04 2022] DELETE /identity/v3/users/38bf0a342fc9441a95bd7087ac7f64a7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:04.903435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:04.903857 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:04.904246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:04.926167 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:04.927705 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:05.004580 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1425/2833] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:19:05.015105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:05.017544 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:05.087676 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a3d4c-2d1d-43d1-8e5f-8fa8800a225b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:05.092563 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:05.093325 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.094015 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.094393 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.105030 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:05.106768 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae865df6-2956-4393-8cc9-7108ddd17254 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:05.118042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1409/2834] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:05.534542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:05.536434 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:05.554065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4db5b-64fb-4ccd-8eca-4a7246296b3f None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:05.565368 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:05.569102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.569690 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.570184 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] PATH_INFO: `/v3/projects/3c29a8669ef14913a64ddfdcc3fcdd09` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.588593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c29a8669ef14913a64ddfdcc3fcdd09)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:05.591351 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:05.625728 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:05.682179 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c29a8669ef14913a64ddfdcc3fcdd09', 'request_id': 'req-11aac17a-7eed-4fa1-849a-3c13e41d4b39'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:05.693615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c29a8669ef14913a64ddfdcc3fcdd09', 'request_id': 'req-11aac17a-7eed-4fa1-849a-3c13e41d4b39'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:05.704916 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c29a8669ef14913a64ddfdcc3fcdd09', 'request_id': 'req-11aac17a-7eed-4fa1-849a-3c13e41d4b39'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:05.723816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] The token cache is being invalidate because project 3c29a8669ef14913a64ddfdcc3fcdd09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:19:05.724100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:05.724765 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-11aac17a-7eed-4fa1-849a-3c13e41d4b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:05.727732 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1426/2835] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:19:05 2022] DELETE /identity/v3/projects/3c29a8669ef14913a64ddfdcc3fcdd09 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:05.733719 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.734181 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.734664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.757769 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:05.759034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:05.844151 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1410/2836] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:05.853529 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.853837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.854116 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.876251 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:05.877336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:05.885854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1427/2837] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:05.890383 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:05.901043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d3825-25ab-4afe-adf0-48fa74de4f57 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:05.906576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:05.907362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.907709 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.908076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.908908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:19:05.909282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:05.911056 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:05.926072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1411/2838] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:19:05.930734 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:05.941051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe154a9-618e-47dc-9558-fe9f2c74bde0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:05.946571 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:05.947321 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:05.947740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:05.948036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:05.949448 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:05.951608 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:05.985603 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1428/2839] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:19:05 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:05.986687 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:05.992171 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:05.999671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59d499-7205-449d-8f11-ed813db1ce75 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:06.072737 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:06.073649 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.074514 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.074787 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.075541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:06.077567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:06.106858 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1412/2840] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:19:05 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:06.108196 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:06.119838 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7f261-5d5f-4964-8d5d-fbde2047aaf8 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:06.125404 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:06.126087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.126461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.126684 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.127427 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:06.129560 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:06.142436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1429/2841] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:19:06.146855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:06.158072 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba16fb-5f98-41a9-928d-c7675a191709 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:06.162973 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:06.163648 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.163941 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.164205 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.164936 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:06.166581 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4cf76-abf4-4cb1-a539-5a9f0f9431ec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:06.182141 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1413/2842] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:06.183971 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:06.198062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b3f8a-4f19-4614-83e6-14f48fbd799e admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:06.205597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:06.207295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.207582 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.207865 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] PATH_INFO: `/v3/projects/810dd4c41fca4b11938e32d08221c9b8/users/d23750f160a445c083ad823e9b2f2d7b/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.228391 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=810dd4c41fca4b11938e32d08221c9b8, user_id=d23750f160a445c083ad823e9b2f2d7b, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:06.231991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:06.252024 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1430/2843] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:06 2022] PUT /identity/v3/projects/810dd4c41fca4b11938e32d08221c9b8/users/d23750f160a445c083ad823e9b2f2d7b/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:19:06.257929 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18599c-ca0b-459a-83c0-78ad989c9561 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.258193 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18599c-ca0b-459a-83c0-78ad989c9561 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.258513 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18599c-ca0b-459a-83c0-78ad989c9561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.280976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1c18599c-ca0b-459a-83c0-78ad989c9561 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:06.282258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1c18599c-ca0b-459a-83c0-78ad989c9561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:06.356131 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1414/2844] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:06.366292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:06.368660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:06.445807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6560a0ba-b514-4ede-b48c-40c8e1731e37 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:06.450332 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:06.451042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:06.451325 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:06.451599 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:06.462275 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:06.464174 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:06.485643 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1431/2845] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:12.471598 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:12.472086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:12.472172 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:12.506372 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] MFA Rules not processed for user `d23750f160a445c083ad823e9b2f2d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:12.508632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:12.599304 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1415/2846] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:12.600475 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:12.612702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cff05-eefe-4c3e-9ed8-df4c5aaecfee service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:12.619271 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:12.620010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:12.620293 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:12.620570 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] PATH_INFO: `/v3/projects` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:12.621288 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:12.623912 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:12.664270 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:12.672036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1432/2847] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:19:12 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:12.676537 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea14814-2acd-44dc-9a3c-44e1dc260bd7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:12.690700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:12.805791 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:12.806986 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:12.807317 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:12.808096 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:12.808835 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:12.812066 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:12.855688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:12.865324 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1416/2848] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:19:12 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:12.869092 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63927-cf27-4458-a956-08f32102b28f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:12.874461 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:12.875233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:12.875517 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:12.875796 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:12.876537 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:12.878295 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:12.901414 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:12.909298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1433/2849] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:12.914882 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be5b4-6eb5-4f93-a6c8-3911578904c8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:12.920235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:12.921306 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:12.921652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:12.922005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] PATH_INFO: `/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e/users/38841dd757db4648a578ceb73b7bfb78/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:12.955338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb8c618e3554b8cb95b7f9c75e51e1e, user_id=38841dd757db4648a578ceb73b7bfb78, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:12.958065 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:12.993487 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1417/2850] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:12 2022] PUT /identity/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e/users/38841dd757db4648a578ceb73b7bfb78/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:12.997407 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:13.023936 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6491f1f-7c9c-4830-abd4-4dac1513f219 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:13.037108 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:13.130301 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:13.131150 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:13.131542 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:13.131896 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:13.132723 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:13.134487 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cf7bb6-39bc-408a-9f59-630a111288f9 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:13.152474 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1434/2851] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:13.153642 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:13.167839 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d13b56b-0cc5-47a2-b579-653e9a0267da admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:13.172794 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:13.173766 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:13.173977 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:13.174232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] PATH_INFO: `/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e/users/38841dd757db4648a578ceb73b7bfb78/roles/2cbc0ba81e9c402a9daa6728ac06f33e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:13.176272 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb8c618e3554b8cb95b7f9c75e51e1e, user_id=38841dd757db4648a578ceb73b7bfb78, role_id=2cbc0ba81e9c402a9daa6728ac06f33e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:13.179447 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:13.193196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1418/2852] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:13 2022] PUT /identity/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e/users/38841dd757db4648a578ceb73b7bfb78/roles/2cbc0ba81e9c402a9daa6728ac06f33e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 13:19:16.268854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f075e-d850-446c-9ea7-609ee0b384eb None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:16.269348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f075e-d850-446c-9ea7-609ee0b384eb None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:16.269348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f075e-d850-446c-9ea7-609ee0b384eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:16.312390 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-ae0f075e-d850-446c-9ea7-609ee0b384eb None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:16.313720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-ae0f075e-d850-446c-9ea7-609ee0b384eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:16.430680 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1435/2853] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:16.448543 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:16.451402 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:16.613433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14effc9-f483-4d26-8978-fc0159194ef4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:16.629390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:16.629390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:16.629390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:16.629390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:16.643364 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:16.645203 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:16.656599 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1419/2854] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:19.560880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:19.564034 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:19.564841 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:19.623659 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] MFA Rules not processed for user `38841dd757db4648a578ceb73b7bfb78`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:19.625100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:19.734359 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1436/2855] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:19.745778 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:19.747956 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:19.808169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395562-34c8-40ba-b490-3a9268845e5a service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:19.812036 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf234ea529642af8039f8e3b0bf601c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:19.812786 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:19.813261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:19.813467 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:19.815699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:19.817651 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:19.821524 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:19.869413 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=100 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:19.890866 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:19.892354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a560-19b5-4f86-aad7-d260821c3c9f service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:19.904500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1420/2856] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:19.933339 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45250d-78fb-4b9d-8f82-1fecf724426d None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:19.939358 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:19.940023 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:19.940386 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:19.940693 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:19.942859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:19.964482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:19.964482 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:19.976711 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1437/2857] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:20.031125 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:20.031421 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:20.031704 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:20.055014 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:20.056258 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:20.065180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1421/2858] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:19:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:20.078097 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:20.092233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef13db-723f-4732-a666-9f383f3be512 service placement] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:20.098333 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:20.099143 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:20.099492 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:20.099839 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:20.111233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:20.113443 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:20.134525 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1438/2859] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:20.206845 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:20.208595 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:20.353338 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7f103-09fa-40fb-8cf2-311f30f3018e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:20.358291 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:20.359423 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:20.359745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:20.359960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] PATH_INFO: `/v3/users/e2f423922c784da2be2d16c736a1ec16` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:20.361180 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f423922c784da2be2d16c736a1ec16)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:20.363214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:20.436376 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f423922c784da2be2d16c736a1ec16'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:20.452058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f423922c784da2be2d16c736a1ec16'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:20.452691 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f423922c784da2be2d16c736a1ec16'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:20.485261 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f423922c784da2be2d16c736a1ec16'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:20.486801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4b243eaf-f436-4e3b-ac6d-d77a8efe74a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f423922c784da2be2d16c736a1ec16'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:20.506784 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1422/2860] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:19:20 2022] DELETE /identity/v3/users/e2f423922c784da2be2d16c736a1ec16 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:20.834126 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:20.859362 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4398d9-1b6f-4a94-9d7d-0730a504e79e service swift] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.006361 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.007203 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.007484 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.007758 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] PATH_INFO: `/v3/projects/3f1179a608f64ede9eb3c4252619f823` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.010906 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1179a608f64ede9eb3c4252619f823)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.013507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.150944 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1179a608f64ede9eb3c4252619f823', 'request_id': 'req-f3086140-6e86-4cc8-924c-a0d677eddf94'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:21.175854 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1179a608f64ede9eb3c4252619f823', 'request_id': 'req-f3086140-6e86-4cc8-924c-a0d677eddf94'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:21.187929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1179a608f64ede9eb3c4252619f823', 'request_id': 'req-f3086140-6e86-4cc8-924c-a0d677eddf94'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:21.209141 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] The token cache is being invalidate because project 3f1179a608f64ede9eb3c4252619f823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:19:21.209736 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:21.210591 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-f3086140-6e86-4cc8-924c-a0d677eddf94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:21.212740 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1439/2861] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:19:20 2022] DELETE /identity/v3/projects/3f1179a608f64ede9eb3c4252619f823 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:21.219586 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.221042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.221507 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.250757 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:21.253480 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:21.277119 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:21.365363 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1423/2862] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:21.367459 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.368200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.368200 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.459194 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:21.464279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:21.501909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1440/2863] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:21.503163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.517534 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.520688 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222e44b-17b2-49f3-8513-bcddd2ab5431 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.521490 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:21.533664 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.534517 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.534795 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.535063 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.535801 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 13:19:21.536105 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.540503 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.570640 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1424/2864] 10.222.0.49 () {58 vars in 1152 bytes} [Thu Jun 30 13:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 13:19:21.573579 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:21.573976 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.596723 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd5082-2d37-4974-b8fe-fce13bfdaf60 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.612237 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.612910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.613184 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.613456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.614155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.618259 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.675163 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1425/2865] 10.222.0.49 () {60 vars in 1152 bytes} [Thu Jun 30 13:19:21 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:21.676359 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.749485 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928840-e36f-4c9e-98c6-3f15f0803651 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.788034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e395bf-afce-4d7d-86c7-2e0a02d49a01 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.803016 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.804868 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.805505 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.808821 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.811373 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.813634 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.867058 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1426/2866] 10.222.0.49 () {60 vars in 1143 bytes} [Thu Jun 30 13:19:21 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 13:19:21.867639 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.890557 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331bf85-f912-481b-b7d2-7feb46a5a659 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.895745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.896422 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.896753 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.897027 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.897937 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.901625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.902146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.902533 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.903056 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.904294 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.906688 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:21.922163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.924014 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:21.930523 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1427/2867] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 13:19:21.932328 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:21.939699 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1441/2868] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:21.970833 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d036c1e-657c-4ebe-a1fb-98f85b5cc0b4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:21.990408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:21.991298 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:21.992226 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:21.992675 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:21.993569 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:21.996076 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c4ea48-cf53-49cf-acaa-6cc2ff786cea admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:22.022995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1428/2869] 10.222.0.49 () {58 vars in 1121 bytes} [Thu Jun 30 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 13:19:22.027625 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:22.046735 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e658b-09b0-4ab0-8fef-980d1644e705 service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:22.061065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:22.062185 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] REQUEST_METHOD: `PUT` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:22.063934 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:22.064412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] PATH_INFO: `/v3/projects/77afad84e1a04d0683cc754a62c51609/users/5e85350136a84c7e866f3246070e58f8/roles/87f5d4315954429eb4783c57c3909faf` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:22.099019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] RBAC: Authorizing `identity:create_grant(project_id=77afad84e1a04d0683cc754a62c51609, user_id=5e85350136a84c7e866f3246070e58f8, role_id=87f5d4315954429eb4783c57c3909faf)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:22.105595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:22.145837 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1442/2870] 10.222.0.49 () {60 vars in 1482 bytes} [Thu Jun 30 13:19:22 2022] PUT /identity/v3/projects/77afad84e1a04d0683cc754a62c51609/users/5e85350136a84c7e866f3246070e58f8/roles/87f5d4315954429eb4783c57c3909faf => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:22.147771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55744d-cdbf-4b12-b12b-1dcce5dfe78e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:22.148060 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55744d-cdbf-4b12-b12b-1dcce5dfe78e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:22.148332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55744d-cdbf-4b12-b12b-1dcce5dfe78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:22.189548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0b55744d-cdbf-4b12-b12b-1dcce5dfe78e None None] MFA Rules not processed for user `5e85350136a84c7e866f3246070e58f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:22.191541 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0b55744d-cdbf-4b12-b12b-1dcce5dfe78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:22.314997 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:22.368638 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1429/2871] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:22.379220 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:22.380963 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:22.504816 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5a3f2-856f-4664-8590-0748f184593f admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:22.517312 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:22.518666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:22.518666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:22.518666 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:22.544710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:22.546102 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ad77f-7779-4907-add2-b14564e638cf service cinder] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:22.556915 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:22.568696 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1443/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:24.299744 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d740e2-2c59-4cb0-99e5-8a43bdbc5342 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:24.300345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d740e2-2c59-4cb0-99e5-8a43bdbc5342 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:24.300345 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d740e2-2c59-4cb0-99e5-8a43bdbc5342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:24.323068 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c7d740e2-2c59-4cb0-99e5-8a43bdbc5342 None None] MFA Rules not processed for user `5e85350136a84c7e866f3246070e58f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:24.324669 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c7d740e2-2c59-4cb0-99e5-8a43bdbc5342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:24.332250 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1430/2873] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:30.664846 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:30.665412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:30.665412 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:30.682592 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] MFA Rules not processed for user `d23750f160a445c083ad823e9b2f2d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:30.683831 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:30.737989 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1444/2874] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:32.945315 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:32.945857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:32.945857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:32.964348 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:32.965578 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:33.022242 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1431/2875] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:19:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:33.035584 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:33.036947 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:33.113932 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be4da8-dec0-40f9-8ccb-b7ca710f3ad6 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:33.120050 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:33.121109 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:33.121394 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:33.121667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:33.130917 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:33.133352 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c454ab-e229-4fc6-9ce6-9254fe2afeba service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:33.141894 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1445/2876] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:43.015198 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=36 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:43.138516 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=57 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:51.232672 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:51.234433 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:51.308625 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e50c5-3407-4f1a-95f1-3c04ee2cb4d4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:51.315101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:51.315762 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:51.316045 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:51.316319 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:51.318302 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:51.330684 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:51.332311 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:51.342504 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1432/2877] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:51.980297 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62c82-9966-4073-8ae0-cd14d30af9a0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:51.980822 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62c82-9966-4073-8ae0-cd14d30af9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:51.980991 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62c82-9966-4073-8ae0-cd14d30af9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:52.044061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-c0e62c82-9966-4073-8ae0-cd14d30af9a0 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:52.045336 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-c0e62c82-9966-4073-8ae0-cd14d30af9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:52.065159 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1446/2878] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:19:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:52.078134 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:52.099185 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6229-a7b0-4d80-8230-ed8423f91bef service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:52.104043 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:52.104682 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:52.104963 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:52.105235 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:52.113323 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:52.115155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:52.123146 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1433/2879] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:52.655702 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3be93-f79b-4144-a6eb-360284ea1884 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:52.656062 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3be93-f79b-4144-a6eb-360284ea1884 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:52.656406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3be93-f79b-4144-a6eb-360284ea1884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:52.677186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-b1d3be93-f79b-4144-a6eb-360284ea1884 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:52.678515 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b1d3be93-f79b-4144-a6eb-360284ea1884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:52.686059 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1447/2880] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:52.700749 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:52.712150 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a299085-7aea-4a69-b74b-cabb512ea8bf service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:52.717527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:52.718545 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:52.718859 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:52.719101 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:52.729002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:52.730643 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:52.738884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1434/2881] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:52.947774 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235833c-79e7-46e8-93e5-561228927263 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:52.948065 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235833c-79e7-46e8-93e5-561228927263 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:52.948365 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235833c-79e7-46e8-93e5-561228927263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:52.967658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-8235833c-79e7-46e8-93e5-561228927263 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:52.969205 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-8235833c-79e7-46e8-93e5-561228927263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:52.976715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1448/2882] 10.222.0.49 () {58 vars in 1009 bytes} [Thu Jun 30 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:19:52.988800 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:52.999861 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221c4b-08aa-434f-8081-248b47dd2306 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:53.005051 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:53.005731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:53.005996 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:53.006268 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:53.014617 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:53.016230 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:53.024332 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1435/2883] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:53.366999 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:53.367486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:53.367594 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:53.387501 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:53.388626 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:53.397494 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1449/2884] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:19:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:19:53.409793 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:53.421155 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981af03-3466-4b27-9383-1bc643c2fed4 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:53.426926 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:53.427727 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:53.427895 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:53.428333 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:53.436717 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:53.438354 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0308a8-5bec-42e4-951f-ff55ab358e9b service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:53.446981 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1436/2885] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:19:55.787807 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:55.853019 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca631a-2eaa-4906-b720-7d35ecde6be0 None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:55.858186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:55.859228 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:55.859595 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:55.859892 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] PATH_INFO: `/v3/users/5e85350136a84c7e866f3246070e58f8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:55.861114 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e85350136a84c7e866f3246070e58f8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:55.863171 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:55.914677 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e85350136a84c7e866f3246070e58f8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:55.924379 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e85350136a84c7e866f3246070e58f8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:55.925163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e85350136a84c7e866f3246070e58f8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:55.938010 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e85350136a84c7e866f3246070e58f8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:55.938803 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e85350136a84c7e866f3246070e58f8'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:55.961435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1450/2886] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:19:55 2022] DELETE /identity/v3/users/5e85350136a84c7e866f3246070e58f8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:19:55.961597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:55.961597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:55.961597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:55.980390 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] MFA Rules not processed for user `b45b25b709bf447ca67a413397143f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:19:55.981903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:56.049605 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1437/2887] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 13:19:56.059597 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:56.061181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:56.075764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:56.124190 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaeaa0a-8aa6-4b62-b0bb-c5cf2c1c6d1b admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:56.128406 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:56.129061 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:56.129344 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:56.129616 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:56.139233 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:56.140691 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:56.150930 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1451/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:19:56.328743 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:19:56.330214 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:19:56.342700 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f736319f-754b-4eb4-b57d-d64e7429f9d3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:19:56.348959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:19:56.349964 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:19:56.350305 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:19:56.350600 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] PATH_INFO: `/v3/projects/77afad84e1a04d0683cc754a62c51609` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:19:56.351897 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77afad84e1a04d0683cc754a62c51609)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:19:56.354567 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:19:56.380619 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:19:56.408576 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77afad84e1a04d0683cc754a62c51609', 'request_id': 'req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:56.417798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77afad84e1a04d0683cc754a62c51609', 'request_id': 'req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:56.424408 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77afad84e1a04d0683cc754a62c51609', 'request_id': 'req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:56.435295 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] The token cache is being invalidate because project 77afad84e1a04d0683cc754a62c51609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:19:56.435602 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:56.436247 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:19:56.437805 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1438/2889] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:19:56 2022] DELETE /identity/v3/projects/77afad84e1a04d0683cc754a62c51609 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:20:05.525925 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:20:05.527700 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:20:05.543706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975840-2340-4af7-835d-d8689eba5208 service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:20:05.602226 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:20:05.603006 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:20:05.603342 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:20:05.603615 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:20:05.605707 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:20:05.671945 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:20:05.673387 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:20:05.683859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1452/2890] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:20:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:20:06.432256 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:20:15.678937 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=71 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:21:10.084290 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:21:10.085883 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:10.100106 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abc6b1-0169-4a9e-8d8b-424138fe1c46 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:21:10.104885 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:21:10.105368 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:10.105652 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:10.105961 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:10.107960 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:10.186456 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:21:10.187842 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:21:10.199980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1439/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:21:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:21:11.413844 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:21:11.415678 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:11.427706 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257ca5d-6850-44c1-83fd-1aa98c1f95cb service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:21:11.431872 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:21:11.432499 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:11.432770 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:11.433042 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:11.435165 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:11.450630 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:21:11.452710 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bba8b-647f-4195-9638-608fd4cf85e2 service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:21:11.468284 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1453/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:21:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:21:11.932659 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:21:11.934959 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:12.000094 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b080e-efa2-4440-a0df-79d8b1a3b43d service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:21:12.004282 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '607bae995849489db891582db395e28c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:21:12.004947 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:12.005225 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:12.005490 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:12.007913 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:12.068594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:21:12.070035 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:21:12.080710 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1440/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:21:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:21:21.462527 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=6 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:21:22.074978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=82 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:21:23.169136 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] REQUEST_METHOD: `POST` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:23.169715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:23.169715 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:23.187551 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.auth.core [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] MFA Rules not processed for user `1c4aa9e4d27c4c2ea4b49ab188f841da`. Rule list: `[]` (Enabled: `True`). {{(pid=80101) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:21:23.188764 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:23.196262 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1454/2894] 10.222.0.49 () {58 vars in 957 bytes} [Thu Jun 30 13:21:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:21:23.536429 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:21:23.546472 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aabad3-9316-4321-9530-0f8d0810d369 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:21:23.550257 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:21:23.550918 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:23.551196 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:23.551475 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:23.553341 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:23.563755 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:21:23.565183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a53c5a-c02a-41b3-904a-30ac302aa9c6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:21:23.573527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1441/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:21:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:21:33.182898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=3 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:21:33.569782 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:21:40.958279 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:21:40.969458 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed3583-1416-41de-8dfe-faba977656fc None None] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:21:40.974049 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:21:40.974722 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:21:40.975078 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:21:40.975246 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:21:40.977256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:21:41.045095 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:21:41.046552 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:21:41.056785 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1455/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:21:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:21:51.052382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=30 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:22:41.060362 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:41.060857 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:41.060982 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:41.079136 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] MFA Rules not processed for user `1bc929bde94d48e2b48e959d970ab1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 13:22:41.080340 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:41.139374 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1442/2897] 10.222.0.49 () {58 vars in 967 bytes} [Thu Jun 30 13:22:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 13:22:41.199988 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:41.201598 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:41.286396 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd687f74-b73f-4b06-a168-c562aba0ef7a service neutron] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:41.291802 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b2d6693e40d40f988254bbc7fcca45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:41.292451 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:41.292762 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:41.293038 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:41.301806 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:41.303435 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:41.311953 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1456/2898] 10.222.0.49 () {60 vars in 1384 bytes} [Thu Jun 30 13:22:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 13:22:48.142233 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:48.143725 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:48.209674 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3d67-6df2-4faf-b3c7-d2e2e7edb88f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:48.215855 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:48.216775 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:48.217399 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:48.217671 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] PATH_INFO: `/v3/users/d23750f160a445c083ad823e9b2f2d7b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:48.218945 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23750f160a445c083ad823e9b2f2d7b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:48.220884 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:48.275921 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23750f160a445c083ad823e9b2f2d7b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.285326 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23750f160a445c083ad823e9b2f2d7b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.285908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23750f160a445c083ad823e9b2f2d7b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.299554 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23750f160a445c083ad823e9b2f2d7b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.300695 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23750f160a445c083ad823e9b2f2d7b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.313685 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1443/2899] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:22:48 2022] DELETE /identity/v3/users/d23750f160a445c083ad823e9b2f2d7b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:22:48.496862 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:48.508274 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd084b0-482c-43a3-9393-9f4bcc46c20f service swift] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:48.566486 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:48.567355 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:48.567636 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:48.567909 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] PATH_INFO: `/v3/projects/810dd4c41fca4b11938e32d08221c9b8` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:48.569087 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=810dd4c41fca4b11938e32d08221c9b8)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:48.571068 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:48.621111 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810dd4c41fca4b11938e32d08221c9b8', 'request_id': 'req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.629795 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810dd4c41fca4b11938e32d08221c9b8', 'request_id': 'req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.638957 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810dd4c41fca4b11938e32d08221c9b8', 'request_id': 'req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.651036 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] The token cache is being invalidate because project 810dd4c41fca4b11938e32d08221c9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:22:48.651319 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.652002 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.653569 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1457/2900] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:22:48 2022] DELETE /identity/v3/projects/810dd4c41fca4b11938e32d08221c9b8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:22:48.657745 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:48.659400 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:48.672542 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b13f0-795a-4921-9317-0eff224d4cc9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:48.693002 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:22:48.733358 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:48.734238 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:48.734548 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:48.734823 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] PATH_INFO: `/v3/users/38841dd757db4648a578ceb73b7bfb78` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:48.736042 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38841dd757db4648a578ceb73b7bfb78)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:48.738183 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:48.790370 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38841dd757db4648a578ceb73b7bfb78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.803637 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38841dd757db4648a578ceb73b7bfb78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.804283 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38841dd757db4648a578ceb73b7bfb78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.817309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38841dd757db4648a578ceb73b7bfb78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.818001 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38841dd757db4648a578ceb73b7bfb78'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:48.831367 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1444/2901] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:22:48 2022] DELETE /identity/v3/users/38841dd757db4648a578ceb73b7bfb78 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:22:49.010276 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:49.011941 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:49.025507 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd4fcd-5f82-4a95-bf59-a4820a287fe0 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:49.030617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:22:49.088161 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:49.089020 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:49.089292 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:49.089559 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] PATH_INFO: `/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:49.090754 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb8c618e3554b8cb95b7f9c75e51e1e)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:49.093018 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:49.143256 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb8c618e3554b8cb95b7f9c75e51e1e', 'request_id': 'req-2aac11ad-2dee-41db-8a59-db2c3e99d142'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:49.152859 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb8c618e3554b8cb95b7f9c75e51e1e', 'request_id': 'req-2aac11ad-2dee-41db-8a59-db2c3e99d142'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:49.159324 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb8c618e3554b8cb95b7f9c75e51e1e', 'request_id': 'req-2aac11ad-2dee-41db-8a59-db2c3e99d142'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:49.170577 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] The token cache is being invalidate because project 2fb8c618e3554b8cb95b7f9c75e51e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80101) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:22:49.170874 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:49.171618 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:22:49.173416 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1458/2902] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:22:49 2022] DELETE /identity/v3/projects/2fb8c618e3554b8cb95b7f9c75e51e1e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:22:58.827255 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:22:59.167327 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:22:59.353686 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:22:59.355394 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:59.422005 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea91dab-e7c3-492a-a87b-90544bb2c62c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:22:59.476632 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:22:59.477289 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:22:59.477572 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:22:59.477847 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:22:59.479978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:22:59.490207 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:22:59.491715 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:22:59.500309 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1445/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:22:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:23:09.496500 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=46 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:23:32.739436 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:32.741428 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:32.836895 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac11ad-2dee-41db-8a59-db2c3e99d142 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:32.844519 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a0b0da6c54341ff8fb66f48a0a32b70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:32.846000 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:32.846647 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:32.846923 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:32.848931 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:32.864880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:32.866658 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:32.879462 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1459/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:23:42.873819 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=34 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:23:45.074597 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:45.076071 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:45.086773 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee88b4-beee-43da-b588-ca6e1f5ef365 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:45.090531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e59c5b4dfe04f30a195ab3e83f49c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:45.091177 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:45.091457 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:45.091731 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:45.093908 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:45.158592 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:45.160040 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:45.169995 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1446/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:23:51.559013 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:51.560505 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:51.575720 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d948e-8c1b-43e8-bd01-8383c68598df service nova] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:51.580181 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:51.581051 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:51.581331 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:51.581629 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] PATH_INFO: `/v3/users/1c4aa9e4d27c4c2ea4b49ab188f841da` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:51.583119 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4aa9e4d27c4c2ea4b49ab188f841da)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:51.585040 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:51.648298 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4aa9e4d27c4c2ea4b49ab188f841da'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:51.658163 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4aa9e4d27c4c2ea4b49ab188f841da'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:51.659043 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4aa9e4d27c4c2ea4b49ab188f841da'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:51.674830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4aa9e4d27c4c2ea4b49ab188f841da'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:51.675498 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4aa9e4d27c4c2ea4b49ab188f841da'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:51.688617 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1460/2906] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:23:51 2022] DELETE /identity/v3/users/1c4aa9e4d27c4c2ea4b49ab188f841da => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:23:51.874497 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:51.884751 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609a5e5-f521-4282-bcb6-f67a83b37c9d service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:51.943980 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:51.944950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:51.945234 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:51.945508 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] PATH_INFO: `/v3/projects/80ff68a0b07e43d4bdb9c32f7166c9a1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:51.946910 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ff68a0b07e43d4bdb9c32f7166c9a1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:51.948853 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:51.997771 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ff68a0b07e43d4bdb9c32f7166c9a1', 'request_id': 'req-5a638246-eb7e-4158-8401-2ca600098d9e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.007527 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ff68a0b07e43d4bdb9c32f7166c9a1', 'request_id': 'req-5a638246-eb7e-4158-8401-2ca600098d9e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.013954 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ff68a0b07e43d4bdb9c32f7166c9a1', 'request_id': 'req-5a638246-eb7e-4158-8401-2ca600098d9e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.024562 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] The token cache is being invalidate because project 80ff68a0b07e43d4bdb9c32f7166c9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:23:52.024903 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.025566 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.027197 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1447/2907] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:23:51 2022] DELETE /identity/v3/projects/80ff68a0b07e43d4bdb9c32f7166c9a1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:23:52.031370 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:52.032830 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:52.046241 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbec3a-fc09-4d44-872f-466260aef658 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:52.110186 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:52.111100 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:52.111376 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:52.111653 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] PATH_INFO: `/v3/users/2eed55ce903b45628fc28a9e747b235d` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:52.112893 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eed55ce903b45628fc28a9e747b235d)` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:52.114880 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:52.164124 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eed55ce903b45628fc28a9e747b235d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.171756 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eed55ce903b45628fc28a9e747b235d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.172382 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eed55ce903b45628fc28a9e747b235d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.184348 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eed55ce903b45628fc28a9e747b235d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.185070 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.notifications [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eed55ce903b45628fc28a9e747b235d'} {{(pid=80101) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.187176 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:23:52.200337 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1461/2908] 10.222.0.49 () {58 vars in 1223 bytes} [Thu Jun 30 13:23:52 2022] DELETE /identity/v3/users/2eed55ce903b45628fc28a9e747b235d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:23:52.378971 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:23:52.380531 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:23:52.394034 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a638246-eb7e-4158-8401-2ca600098d9e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:23:52.423232 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:23:52.463705 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b45b25b709bf447ca67a413397143f8d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '288f97ff4fc34a4e979d3c926f4a9c7f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:23:52.464789 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:23:52.465169 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:23:52.465529 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] PATH_INFO: `/v3/projects/44f7c71ddce64c24acf7eea84534609b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:23:52.467594 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f7c71ddce64c24acf7eea84534609b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:23:52.470655 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:23:52.534300 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f7c71ddce64c24acf7eea84534609b', 'request_id': 'req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.545950 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f7c71ddce64c24acf7eea84534609b', 'request_id': 'req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.552978 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f7c71ddce64c24acf7eea84534609b', 'request_id': 'req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.563820 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] The token cache is being invalidate because project 44f7c71ddce64c24acf7eea84534609b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 13:23:52.564123 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.564798 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b82f6b9-2995-44e3-9ab4-6ede7505b2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 13:23:52.566452 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1448/2909] 10.222.0.49 () {58 vars in 1232 bytes} [Thu Jun 30 13:23:52 2022] DELETE /identity/v3/projects/44f7c71ddce64c24acf7eea84534609b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 13:24:02.197129 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=4 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:24:02.561086 dv-cinder-812685-14-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 13:25:01.875660 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Authenticating user token {{(pid=80101) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 13:25:01.877836 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:25:01.957778 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad0b2e-c182-444a-9318-015d62b4fae4 admin admin] Validating token access rules against request {{(pid=80101) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 13:25:02.021372 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '998f1e616d984e7fb7feac65a79c5599', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a198bc016a3c49ae92f7ee4ae3dbd311', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80101) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 13:25:02.022296 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] REQUEST_METHOD: `GET` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 13:25:02.022620 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 13:25:02.022898 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80101) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 13:25:02.024929 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.fernet_utils [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80101) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 13:25:02.036704 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 13:25:02.038593 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd1e2db-5832-4bf8-91d9-1db0ef2ea8a4 service placement] RBAC: Authorization granted {{(pid=80101) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 13:25:02.048667 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: [pid: 80101|app: 0|req: 1462/2910] 10.222.0.49 () {60 vars in 1365 bytes} [Thu Jun 30 13:25:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 13:25:12.044521 dv-cinder-812685-14-iscsi devstack@keystone.service[80101]: DEBUG dbcounter [-] [80101] Writing DB stats keystone:SELECT=58 {{(pid=80101) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}